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ABSTRACT 


One  of  the  most  important  missions  all  Navies  have  is  to  constantly  and 
sufficiently  monitor  their  area  of  responsibility.  This  task  becomes  more 
challenging  when  a  surveillance  system  operates  in  a  complex  environment  with 
high  traffic  of  merchant  and  fishing  vessels  and  the  existence  of  many  islands. 
Potential  tactics  that  targets  might  use  increase  the  difficulty  of  this  task. 
Integrating  Unmanned  Aerial  Vehicles  (UAVs)  into  a  surveillance  system  that 
consists  of  ground  radars  and  surface  ships  might  enhance  the  system’s 
capabilities  and  mitigate  its  vulnerabilities.  In  this  study,  the  extremely  complex 
maritime  environment  of  the  Aegean  Sea  is  modeled  in  the  Map  Aware  Non 
Uniform  Automata  (MANA)  agent-based  simulation  environment  to  explore  the 
effectiveness  of  UAVs  in  those  conditions.  The  results  from  almost  100,000 
simulated  Intelligence,  Surveillance,  and  Reconnaissance  missions  are  analyzed 
using  descriptive  statistics,  ANOVA,  stepwise  regression,  and  partition  trees.  It 
was  found  that  by  integrating  one  or  two  UAVs  into  a  traditional  surveillance 
system,  it  becomes  more  efficient  in  the  detection  and  persistent  surveillance  of 
enemies  and  neutral  targets.  The  most  important  factors  that  affect  the 
surveillance  system’s  performance  are  the  detection  capabilities  of  its  sensors, 
the  communication  accuracy,  and  the  enemy’s  counter-detection  capability. 
Thus,  Greece  and  other  countries  with  similar  geographical  characteristics 
should  deploy  UAVs  in  a  maritime  surveillance  role. 
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been  made  within  the  time  available  to  ensure  that  the  programs  are  free  of 
computational  and  logical  errors,  they  cannot  be  considered  validated.  Any 
application  of  these  programs  without  additional  verification  is  at  the  risk  of  the 
user. 
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EXECUTIVE  SUMMARY 


One  of  the  most  fundamental  missions  of  the  Hellenic  Navy  is  surveillance  of  its 
territorial  waters,  as  well  as  the  waters  in  its  region.  This  mission  is  undoubtedly 
of  strategic  importance  taking  into  consideration  the  geographical  region  where 
Greece  is  located,  at  the  intersection  of  three  continents  (Europe,  Asia  and 
Africa),  as  well  as  Greece’s  participation  in  the  European  Union  (EU)  and  North 
Atlantic  Treaty  Organization  (NATO). 

Monitoring  the  Aegean  Sea,  the  Eastern  area  of  the  Mediterranean  Sea,  is 
by  default  difficult  because  it  demands  cooperation  among  different  sensors. 
These  sensors  might  be  ground  radars,  patrol  ships,  air  force  reports,  and 
various  electromagnetic  or  electronic  detections.  The  complexity  of  the  maritime 
environment  increases  the  difficulty  in  the  surveillance.  The  most  distinctive 
geographical  characteristic  of  the  Aegean  Sea  is  the  number  of  islands  located 
there.  Despite  its  relatively  narrow  area,  the  Aegean  Sea  accommodates  more 
than  3,000  islands,  islets,  and  rocks.  Furthermore,  this  sea  is  used  by 
approximately  55,000  transiting  merchant  vessels  on  an  annual  basis. 

The  current  maritime  surveillance  system  in  Greece,  despite  its  cost  and 
its  demands  on  human  resources  and  material,  is  highly  vulnerable  due  to  a 
number  of  factors.  First  of  all,  the  weather  conditions  affect  the  current  sensors  in 
various  ways.  For  example,  high  sea  states  may  prevent  small  surface  ships 
from  patrolling.  Moreover,  low  visibility  does  not  allow  the  visual  observers  to 
locate  targets  at  longer  distances.  Similarly,  in  rainy  conditions  radars  are 
negatively  affected,  resulting  in  smaller  detection  ranges.  Another  limiting  factor 
is  location.  “Blind  sectors,”  the  areas  that  are  not  covered  by  the  ground  radar 
systems,  result  from  the  way  systems  must  be  positioned.  Finally,  the 
adversary’s  tactics  may  diminish  the  detection  range.  Such  tactics  include  two 
targets  sailing  very  close  to  one  another  or  vessels  staying  close  to  the  shoreline. 
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Greece  is  facing  a  major  challenge  to  cope  with  the  large  influx  of  mixed 
migratory  flows  (including  irregular  migrants,  refugees,  and  asylum  seekers)  and 
the  current  economic  crisis.  According  to  EU,  Greece  has  become  the  main  entry 
point  for  irregular  migratory  flows  into  the  European  Union.  In  2010,  more  than 
132,000  third-country  nationals  were  arrested  in  Greece,  including  53,000  in  the 
Greek-Turkish  border  regions.  During  the  first  ten  months  of  2012,  over  70,000 
arrests  occurred,  including  about  32,000  at  the  border  with  Turkey.  People  came 
from  110  different  countries,  the  majority  from  Asia  and  the  Middle-East. 

Conventional  wisdom  states  that  Unmanned  Aerial  Vehicles  (UAVs)  offer 
two  main  advantages  over  manned  aircraft:  they  are  considered  more  cost- 
effective,  and  they  minimize  the  risk  to  a  pilot’s  life.  This  study  investigates  the 
ways  in  which  UAVs  can  be  integrated  into  a  traditional  surveillance  system  that 
utilizes  only  ground  radars  and  surface  ships. 

Using  an  agent-based  simulation  environment  called  Map-Aware  Non- 
uniform  Automata  (MANA),  two  representative  areas  of  the  Aegean  Sea  are 
modeled  for  our  research;  the  “Open  Sea”  model,  where  a  ship  has  to  sail  for 
about  50  nautical  miles  without  the  “coverage”  of  nearby  islands,  and  the  “Many 
Islands”  model,  where  several  islands  are  very  close  to  one  another.  The  first 
represents  a  typical  area  of  the  Northern  and  Southern  Aegean  Sea  and  the 
second  of  the  Central  and  Southeast  Aegean  Sea.  These  rare  geographical 
conditions  compose  a  challenging  environment  operationally. 

In  our  simulation  the  Red  Force  consists  of  four  surface  vessels.  These 
vessels  attempt  to  transport  illegal  immigrants  into  Blue’s  area  of  responsibility. 
To  increase  the  possibility  of  going  undetected,  they  keep  close  to  nearby 
merchant  vessels  that  have  approximately  the  same  course  with  them  (“Open 
Sea”  model),  or  they  try  to  take  advantage  of  islands  by  sailing  close  to  the 
shoreline  during  their  route  (“Many  Islands”  model). 

Following  the  base  model,  a  Nearly  Orthogonal  Latin  Hypercube  (NOLH) 
is  created  to  efficiently  explore  the  effect  of  21  variables  on  the  percentage  of 
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time  the  targets  are  positively  identified.  These  21  factors  were  chosen  based  on 
opinions  of  subject  matter  experts,  as  well  as  the  author’s  experience.  In  addition 
to  the  21  independent  variables  that  are  explicitly  varied  in  the  Design  of 
Experiment  (DOE),  there  are  an  additional  52  variables  that  depend  on  one  of 
the  aforementioned  factors.  These  variables  describe  the  sensors’  capabilities. 

The  simulation  runs  show  that  integrating  one  or  two  UAVs  into  a 
traditional  surveillance  system  makes  it  more  efficient  in  the  detection  and 
persistent  surveillance  of  enemies  and  neutral  targets.  This  conclusion  applies  to 
all  the  areas  of  the  Aegean  Sea,  both  the  Northern  and  Southern  Aegean  Sea 
(modeled  by  the  “Open  Sea”  model)  and  the  Central  and  Southeast  (modeled  by 
the  “Many  Islands”  model). 

In  all  the  cases  we  studied,  we  consistently  found  that  a  traditional  system 
with  only  surface  ships  and  ground  radars  performs  worse  than  a  system  that 
uses  UAVs.  Additionally,  we  explored  a  “futuristic”  scenario  in  which  only  two 
UAVs  support  the  ground  radars,  without  the  use  of  surface  ships  (no  Blue  ships 
scenario).  The  results  show  that  we  can  obtain  similar  results  with  the  case 
where  we  deploy  one  UAV  and  two  surface  ships  in  support  of  the  ground  radars 
(1-UAV  scenario).  This  conclusion  indicates  that  decision  makers  should  review 
the  surveillance  policy.  They  can  shift  from  the  cost-ineffective  use  of  surface 
ships  towards  the  use  of  UAVs  to  better  monitor  maritime  areas.  This  solution 
might  be  more  beneficial  in  adverse  weather  conditions  or  when  there  is  a  lack  of 
operational  surface  units. 

This  study  also  demonstrates  that  the  potential  tactics  the  enemy  might 
use  to  avoid  detection  cannot  mislead  UAVs.  Whether  the  enemies  try  to  take 
advantage  of  nearby  merchant  vessels  and  sail  close  to  them  to  transit  long 
distances,  or  to  use  the  shorelines  of  numerous  islands  as  coverage,  a 
surveillance  system  that  deploys  UAVs  is  able  to  present  significantly  better 
performance  than  a  system  that  uses  only  ground  radars  and  surface  ships. 


Furthermore,  this  thesis  shows  that  the  area  from  which  a  UAV  is 
launched  affects  the  performance  of  the  whole  surveillance  system.  As  it  is  easily 
understood,  the  more  contacts  that  exist  close  to  the  launch  site,  the  more  the 
UAV  contributes  to  the  surveillance  system.  That  is,  if  the  launch  site  is  close  to 
areas  with  high  traffic,  the  UAV  will  have  more  opportunities  to  locate  the  targets. 
Additionally,  the  targets  will  be  identified  earlier.  Therefore,  it  is  extremely 
important  to  define  the  best  launch  site  for  the  UAVs.  From  the  literature  review, 
we  found  that  UAVs  can  be  launched  either  from  ground-based  sites  or 
appropriately  equipped  surface  ships.  Both  solutions  must  be  examined  for 
strengths  and  vulnerabilities. 

This  study  also  shows  that  UAVs  can  mitigate  a  vulnerability  that  ground 
radar  systems  inherently  have:  the  inability  for  fixed  radars  to  cover  their  “blind 
sectors,”  which  result  from  the  way  systems  must  be  positioned.  Because 
monitoring  the  whole  range  of  the  Area  of  Interest  (AOI)  with  ground  radars  is  not 
practical  (it  would  require  twice  as  many  sensors),  the  existing  radars  are  placed 
in  areas  with  the  most  traffic.  As  a  consequence,  these  systems  are  not  always 
stationed  at  the  highest  point  of  an  island,  which  results  in  a  limited  field  of  view. 
Additionally,  taking  into  account  that  radar  technology  is  subject  to  technical 
restrictions,  the  detecting  capabilities  are  further  narrowed.  UAVs  can  easily 
cover  those  areas  providing  information  about  existing  targets. 

Finally,  this  thesis  found  the  important  factors  that  contribute  to  the 
maritime  surveillance  system.  The  two  models  that  modeled  corresponding 
environments  in  terms  of  transiting  merchant  vessels,  the  existence  of  nearby 
islands  and  the  enemy’s  tactics  were  studied  separately,  but  most  of  the  input 
variables  appear  in  both  of  them. 

•  The  detection  range  of  all  the  sensors  that  are  part  of  the 
surveillance  system  seems  to  have  the  most  significant  positive 
effect  on  the  percentage  of  time  the  targets  are  classified.  The 
partition  tree  for  the  “Open  Sea”  model  highlights  the  importance  of 
having  the  radars,  the  UAV,  and  the  ships’  detection  ranges  larger 
than  22,  19  and  16  km,  respectively. 


XXIV 


•  Communication  accuracy  of  the  Operational  Centre  has  a 
significant  positive  effect  on  the  system’s  performance.  The 
partition  tree  for  the  “Many  Islands”  model  suggests  a  value  of  at 
least  78  percent  to  better  monitor  the  area  of  interest.  It  worth 
noticing  that  the  Operational  Centre  is  crucial  in  the  communication 
network  that  we  established  because  its  role  is  to  gather  the 
information  from  each  one  of  the  sensors  and  send  them  to  the 
patrolling  units. 

•  Enemy’s  stealth  is  a  factor  that  has  a  significant  negative  impact  on 
the  percentage  of  time  a  target  is  classified.  Enemy’s  stealth  is 
presented  in  both  the  scenarios  we  studied  where  a  target  used  a 
specific  tactic  to  avoid  being  detected  by  the  surveillance  system.  In 
reality,  this  factor  describes  the  enemy’s  attempt  to  avoid  the  visual, 
infrared  (IR),  and  radar  contact  using  various  methods. 

•  The  number  of  merchant  vessels  and  their  speed  has  a  negative 
effect  on  the  surveillance  system’s  performance  only  in  the  case 
where  the  enemy  tries  to  use  them  as  a  “Trojan  Horse”  to  transit 
long  distances.  In  environments  where  there  is  high  traffic,  either 
from  merchant  or  fishing  vessels,  but  the  enemy  does  not  try  to 
take  advantage  of  it,  it  seems  that  the  surveillance  system  is  not 
affected. 

•  Average  Times  between  Detections  of  our  own  sensors  are 
additional  important  factors  in  terms  of  the  surveillance  system’s 
detection  capability.  All  of  them  positively  affect  the  percentage  of 
time  the  targets  are  classified,  regardless  of  the  operational 
environment  in  which  the  system  operates. 

•  The  enemy’s  speed  is  a  factor  that  has  a  significant  negative  effect 
on  the  percentage  of  time  the  targets  are  classified.  This  factor 
emerged  only  in  the  “Open  Sea”  model  where  the  target  has  to 
transit  long  distances  being  “exposed”  to  the  adversary’s 
surveillance  system.  This  is  operationally  reasonable  because 
when  a  target  sails  very  close  to  the  shoreline  to  avoid  the 
detection,  it  cannot  maintain  a  high  speed  due  to  navigation 
dangers. 

•  UAV  speed  has  a  positive  effect  on  the  system’s  performance  only 
in  the  case  where  it  has  to  cover  large  areas.  This  study  showed 
that  when  the  UAV  had  to  cover  areas  with  a  large  number  of 
islands,  its  speed  was  not  important. 
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I.  INTRODUCTION 


A.  GENERAL  FRAMEWORK 

1.  The  Role  of  Hellenic  Navy  in  Surveillance 

The  geographical  region  where  Greece  is  located,  at  the  intersection  of 
three  continents  (Europe,  Asia  and  Africa),  is  undoubtedly  of  strategic  importance 
in  surveillance  (Figure  1).  Traditionally,  Greece’s  foreign  policy  focuses  on  two 
major  priorities.  The  first  priority  is  to  employ  foreign  policy  as  “an  operational 
instrument  for  solving  problems  both  in  the  region,  where  Greece  is  located,  as 
well  as  in  the  wider  international  system.” 


Figure  1 .  Strategic  importance  of  Greece’s  location  (From  Wikimedia.org). 

The  second  priority  is  “to  contribute  actively  to  promoting  peace,  stability 
and  cooperative  patterns  of  behavior  in  the  region  of  Southeastern  Europe  and 
most  specifically  in  the  Balkans,  and  the  area  of  the  Eastern  Mediterranean” 
(Kranidiotis,  1998). 
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Working  in  this  context,  the  Hellenic  (Greek)  Navy's  mission  includes 
“contribution  to  the  deterrence  against  any  potential  aggressor  and  the  unabated 
support  to  Allied,  European  Union  and  other  international  efforts  so  as  to  fulfill 
Greece's  commitment  as  a  dedicated  contributor  to  North  Atlantic  Treaty 
Organization,  European  Union,  and  United  Nations,  maintaining  readiness  to 
effectively  contribute  to  defense  and  security  of  the  Nation”  (Karamalikis,  2008, 
P-31). 

One  of  the  most  fundamental  missions  of  the  Hellenic  Navy  is  surveillance 
of  the  territorial  waters,  as  well  as  the  national  waters  that  are  close  to  them. 
This  mission  is  difficult  because  it  demands  cooperation  between  different 
sensors.  These  sensors  might  be  ground  radars,  patrol  ships,  air  force  reports, 
and  various  electromagnetic  or  electronic  detections.  A  serious  parameter  for  the 
Hellenic  Navy  is  the  degree  of  cooperation  with  the  Coast  Guard  and/or  the 
sharing  of  information  with  international  stakeholders. 

The  importance  of  this  mission  is  reinforced  by  Greece’s  participation  in 
the  European  Union  (EU).  In  the  context  of  EU  enlargement,  Europe’s  maritime 
borders  have  now  expanded  with  Europe’s  coastline  to  contain  85  percent  of  the 
EU’s  international  borders.  From  the  EU’s  perspective,  this  expansion  will  require 
increased  surveillance  to  tackle  problems  such  as  illegal  immigration  and  other 
illicit  activities  linked  to  organized  crime,  such  as  drug  smuggling,  human 
trafficking  and  the  trafficking  of  illicit  materials  such  as  WMD  and  explosives. 
Effective  monitoring  of  EU  external  borders  requires  increased  cooperation 
between  relevant  stakeholders,  such  as  coast  guard  organizations  and  law 
enforcement  agencies,  in  maritime  surveillance. 

The  Mediterranean  Basin  is  a  strategically  important  region  for  the  EU, 

and  it  is  necessary  to  construct  a  strong  economic  area  capable  of  contributing  to 

the  Union's  regional  balance  by  assuring  peace,  stability  and  prosperity.  The 

Barcelona  Initiative  was  launched  with  these  objectives  in  mind  and  is  now  a 

central  element  of  the  EU-Mediterranean  Policy.  Furthermore,  the  Hellenic  Navy 

aims  to  be  vigilant  in  detecting  all  threats  that  might  cause  instability  in  the  area 
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in  which  it  operates.  Those  threats,  like  terrorism,  organized  crime,  and  political 
or  economic  crises  must  always  be  taken  into  serious  consideration  at  the 
campaign  level,  as  well  as  at  the  operational  level  (Kyriazis,  2004,  pp.  64-66,  68, 
71-73). 

Traditionally,  the  Navy  handles  the  surveillance  mission  with  multiple 
assets  under  a  broad  mission  called  Surface  Search  and  Control  (SSC).  At  sea, 
surface  detections  are  often  made  by  medium-range  ground  radars  scattered  on 
numerous  islands  which  provide  optimal  coverage  with  respect  to  sensor 
capabilities  and  operational  priorities.  Additionally,  a  large  number  of  surface 
ships  are  assigned  to  patrol  comparatively  small  sectors  in  order  to  contribute  to 
either  the  detection  or  the  identification  task. 

Besides  the  Navy  assets,  ground-based  or  surface,  there  are  additional 
means  from  other  military  branches  or  organizations  that  provide  useful 
information  to  the  Hellenic  Navy  which  has  the  coordinating  role.  A  detailed 
presentation  of  all  the  stakeholders  who  participate  in  the  current  surveillance 
system  is  provided  in  Chapter  II. 

Realistically,  not  all  the  assets  are  available  at  all  times.  All  the  information 
received  from  these  assets  is  used  to  form  and  maintain  the  Recognized 
Maritime  Picture  (RMP).  The  RMP  is  “about  maintaining  an  unambiguous  and 
timely  database  of  the  position  and  identification  of  all  tracks,  both  warship  and 
merchant,  and  being  able  to  distinguish  good  or  cleared  ships  from  the 
adversary,  unchallenged,  suspect,  or  blockade  running  ships”  (Germain,  1997). 
The  RMP  assists  the  commanders  in  obtaining  the  intelligence  needed  to  use 
their  assets  more  effectively  and  consequently  achieve  their  mission. 

2.  Complexity  of  the  Maritime  Environment 

The  Aegean  Sea,  which  is  the  Eastern  maritime  frontier  of  European 
Union,  is  the  area  in  which  the  Hellenic  Navy  mostly  operates.  The  most 
distinctive  geographical  characteristic  of  Aegean  Sea  is  undoubtedly  the  number 
of  islands  located  there.  Despite  its  relatively  narrow  area,  the  Aegean  Sea 
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accommodates  more  than  3,000  islands,  islets  and  rocks  (Acer,  2003). 
Furthermore,  this  sea  is  used  by  transiting  merchant  vessels  sailing  towards  the 
Black  Sea  (Northeast),  Red  Sea  (South)  and  Middle  East  (East).  It  is  estimated 
that  approximately  55,000  merchant  vessels  cross  this  area  annually  (Camci, 
Eldemir,  Uysal,  &  Ustun,  2009,  pp.  424-429).  If  we  take  into  consideration  the 
large  number  of  fishing  vessels  (both  Greek  and  Turkish)  and  sailing  boats, 
especially  during  the  summer,  one  can  conclude  how  difficult  it  is  to  monitor  this 
Area  of  Interest  (AOI).  This  task  is  further  complicated  when  a  “target”  uses 
unusual  tactics  to  mislead  the  assigned  surveillance  sensors. 

3.  Vulnerabilities  of  the  Current  Surveillance  System 

The  current  maritime  surveillance  system  in  Greece,  despite  its  cost  and 
its  demands  on  human  resources  and  material,  is  highly  vulnerable  due  to  a 
number  of  factors.  First  of  all,  the  weather  conditions  affect  the  current  sensors  in 
various  ways.  For  example,  high  sea  state  prevents  small  surface  ships  from 
patrolling.  Moreover,  low  visibility  does  not  allow  the  visual  observers  to  locate 
targets  at  large  distances.  Similarly,  in  rainy  conditions  radars  are  negatively 
affected,  resulting  in  smaller  detection  ranges  (Schneider  &  Williams,  1977, 
pp.  11-29). 

Another  limiting  factor  is  location.  “Blind  sectors,”  the  areas  that  are  not 
covered  by  the  ground  radar  systems,  result  from  the  way  systems  must  be 
positioned.  Since  monitoring  the  whole  range  of  the  AOI  with  ground  radars  is  not 
practical  (it  would  require  twice  as  many  sensors),  the  existing  radars  are  placed 
in  areas  with  the  most  traffic.  As  a  consequence,  these  systems  are  not  always 
stationed  at  the  highest  point  of  an  island,  which  results  in  a  limited  field  of  view. 
Additionally,  taking  into  account  that  radar  technology  is  subject  to  technical 
restrictions,  the  detecting  capabilities  are  further  narrowed. 

Finally,  the  adversary’s  tactics  may  diminish  the  detection  range.  Such 
tactics  include  two  targets  sailing  very  close  to  one  another  or  close  to  the 
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shoreline.  In  such  cases  the  resolution  of  the  radar  might  not  be  adequate  to 
distinguish  the  targets  (Wehner,  1994;  Hutchinson,  2003). 

Furthermore,  these  limiting  factors  are  multiplied  by  the  complex 
environment  in  which  maritime  activities  occur.  This  complexity  provides  the 
targets  involved  in  illegal  activities  or  adversary  warships  that  are  enemies  of  the 
North  Atlantic  Treaty  Organization  (NATO),  the  EU  or  Greece,  with  potential 
ways  to  avoid  detection. 

As  mentioned  above,  an  extremely  large  number  of  people  and  operating 
subsystems  (units)  have  to  collaborate  in  the  current  surveillance  system.  The 
existing  system  involves  people  from  different  branches  (Navy,  Army,  Air  Force) 
with  different  backgrounds  (personnel  coming  from  conscription  in  contrast  to 
voluntary  military  members),  as  well  as  from  different  countries  and  security 
organizations.  This  diversity  may  be  the  biggest  vulnerability  of  the  surveillance 
system,  even  bigger  than  human  error  or  the  deficiencies  of  the  existing  means. 
Any  new  system  should  take  into  account  the  leveraging  of  differences  among 
participants  to  produce  innovative,  synergistic  solutions  and  balancing  of 
divergent  stakeholder  concerns  (Hardy,  Lawrence,  &  Grant,  2005). 

Adding  to  the  challenge,  although  the  cooperation  with  the  international 
organizations  (NATO,  EU)  is  exceptional,  this  cooperation  cannot  be  exclusively 
relied  upon,  because  the  majority  of  the  RMP  concerning  the  Aegean  Sea 
actually  is  drawn  from  the  Greek  reports.  In  other  words,  the  majority  of  the 
information  flows  from  the  Hellenic  Navy  to  the  other  stakeholders  rather  than  the 
other  way  around.  That  reinforces  Greece’s  responsibility  to  its  allies  for 
providing  near-to-real  time  and  accurate  Situational  Awareness  (SA)  in  the  AOI. 

Furthermore,  the  participation  of  the  EU’s  units  in  patrolling  the  area  of 
operations  definitely  assists  in  the  task  of  surveillance.  Even  so,  the  inherent 
problems  of  that  system,  such  as  vulnerabilities  due  to  weather  conditions, 
fatigue  of  the  crew,  restriction  of  the  detection  devices  (e.g.,  radar,  night  vision 
goggles)  remain  a  challenge  to  the  Greek  Navy.  Additionally,  EU  assistance 
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holds  only  for  a  period  of  time  throughout  the  year  (two  to  three  months 
annually). 

Finally,  the  financial  crisis  in  Greece  resulted  in  a  significant  budget  cut  in 
the  Ministry  of  Defense.  As  a  consequence,  systems  that  are  economical  and,  at 
the  same  time,  highly  effective  must  be  adopted  throughout  all  the  sectors, 
including  the  Hellenic  Navy.  Although  the  exact  cost  for  the  surveillance  of  the 
Aegean  Sea  is  unavailable  and  difficult  to  estimate  (because  it  involves  a 
tremendous  number  of  people  and  manned  devices  or  units),  the  cost  certainly  is 
significant.  Therefore,  budget  cuts  to  surveillance  of  the  Aegean  Sea  prompt  the 
Greek  Navy  to  derive  more  cost  efficient  methods  of  surveillance. 

B.  PROBLEM  STATEMENT 

The  greatest  challenge  for  the  current  surveillance  system,  which  does  not 
involve  Unmanned  Aerial  Vehicles  (UAVs),  is  to  detect  comparatively  small 
targets  (with  small  Radar  Cross-Section).  As  stated  earlier,  a  common  technique 
of  small  targets  is  to  stay  close  to  the  shoreline.  Current  radar  resolution  cannot 
distinguish  the  presence  of  a  small  ship  so  close  to  the  shoreline.  This  problem 
arises  when  hostile,  relatively  small  warships  (like  fast  attack  boats)  or  other 
small  boats,  which  might  participate  in  illegal  transactions  (e.g.,  human 
trafficking,  smuggling,  etc.),  try  to  take  advantage  of  this  situation  and  avoid 
being  detected  by  the  Greek  Naval  surveillance  system. 

A  second  problem  arises  when  small  hostile  warships  are  assigned  to 
transit  an  “open  sea,”  which  is  considered  an  area  with  no  island  for  about  50 
nautical  miles.  In  the  open  sea,  small  ships  are  extremely  exposed  and  can 
readily  be  detected  by  their  adversary’s  assets  (e.g.,  air  assets,  frigates,  ground 
radars,  etc.).  Additionally,  small  ships  are  unable  to  protect  themselves  because 
they  lack  anti-air  weapons  and  self-defense  systems  or  missiles.  Thus,  these 
small  warships  commonly  take  advantage  of  the  presence  of  large  merchant 
vessels/tankers  and  remain  as  close  to  them  as  they  can  to  avoid  detection.  The 
nature  of  the  Aegean  Sea  favors  this  practice. 
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C.  PURPOSE  OF  THIS  STUDY 

The  purpose  of  this  study  is  not  to  build  a  completely  new  surveillance 
system,  but  to  introduce  the  potential  benefits  of  using  new  technologies/sensors 
which  complement  the  current  system.  For  this  reason,  we  will  try  to  explore  the 
ways  in  which  UAVs  can  be  engaged  in  this  process.  This  application  will  be  an 
innovation  for  the  countries  of  the  East  Mediterranean  that  currently  do  not  use 
UAVs  in  surveillance  of  maritime  areas. 

The  primary  reason  that  we  look  into  this  solution  is  because  UAVs  are 
increasingly  utilized  by  developed  countries  in  the  global  operational  environment 
today  (Frederick,  2006),  and  this  trend  is  expected  continue  in  the  years  ahead. 
Additionally,  the  use  of  UAVs  is  believed  to  mitigate  the  inherent  vulnerabilities  of 
the  current  surveillance  system  mentioned  earlier,  as  well  as  reduce  the  overall 
cost  of  surveillance.  Taking  the  U.S.  Navy  as  an  example,  it  intends  to  increase 
the  number  of  UAVs  in  service  while  at  the  same  time  reduce  the  number  of 
operators  (Liu,  Wasson,  &  Vincenzi,  2009,  pp.  795-810).  Another  example  is  the 
replacement  of  helicopters  by  Vertical  Takeoff  Unmanned  Aerial  Vehicles 
(VTUAVs)  in  the  U.S.  Navy's  planned  56-ship  fleet  of  Littoral  Combat  Ships 
(Burgess,  2004,  pp.  24-25). 

Undoubtedly,  the  global  investment  in  unmanned  systems  accelerated 
following  9/11  and  UAV  successes  in  Iraq  and  Afghanistan.  During  the  decade  of 
2005-2015,  the  global  market  for  UAVs  (including  all  air  vehicles,  ground  control 
equipment  and  payloads)  is  an  estimated  $16  billion.  The  U.S.  Department  of 
Defense  (DoD)  annual  profile  for  Unmanned  Aerial  Systems  (UAS)  is  depicted  in 
Figure  2.  Europe  could  spend  over  1  billion  euros  on  procurement  and  possibly 
more  on  research  and  development  (Dickerson,  2007,  pp.  114-1 16). 
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Figure  2.  U.S.  DoD  annual  profile  for  unmanned  aerial  systems  (From  U.S. 

Library  of  Congress,  2012). 

There  have  been  numerous  studies  that  demonstrate  the  effectiveness  of 
UAVs  in  border  security  (Yildiz,  2007)  and  homeland  security  (Myers,  2007; 
Weiger,  2007).  An  additional  fact  that  proves  the  need  for  aerial  means  in  the 
AOI  comes  from  the  NATO  contribution  to  Greece  with  Airborne  Warning  and 
Control  Systems  (AWACS)  aircraft  during  the  2004  Olympic  Games  (Graham, 
2004).  At  that  time,  although  there  was  a  fleet  of  eight  surface  ships  patrolling  the 
Aegean  Sea  (in  national  and  international  waters)  it  was  considered  necessary  to 
have  in  addition  one  air  asset  with  advanced  capabilities.  In  comparison  to  that 
time,  perhaps  the  current  need  for  constant  coverage  of  the  AOI  is  not  so 
obvious,  and  the  assets  assigned  are  definitely  fewer  than  in  the  past.  For  all 
these  reasons  we  will  try  to  answer  similar  questions  from  the  perspective  of 
operating  in  a  complex  maritime  environment. 

This  study  investigates  whether  a  surveillance  system  with  the  use  of 
UAVs  performs  better  than  a  traditional  system.  In  other  words,  we  explore 
whether  UAVs  are  capable  of  mitigating  all  or  part  of  the  vulnerabilities  of  the 
current  sensors  (e.g.,  ground  radars,  patrol  ships,  etc.).  Additionally,  we  explore 


8 


what  factors  should  be  considered  in  designing  more  effective  surveillance 
systems  that  meet  the  needs  of  the  geographical  area  in  which  they  will  operate. 

D.  SCOPE  OF  THESIS 

This  thesis  focuses  on  reinforcing  the  surveillance  system  in  complex 
maritime  environments  where  high  maritime  traffic  is  met  and  where  targets  may 
take  advantage  of  adjacent  islands  or  transiting  merchant  vessels  to  avoid 
detection.  Furthermore,  it  is  assumed  that  the  current  surveillance  system  does 
not  involve  UAV  capabilities,  but  the  benefit  of  UAV  capabilities  will  lead  to  their 
implementation  with  current  sensors  in  the  future. 

The  scope  of  this  study  is  not  to  build  a  new  surveillance  system  from  the 
beginning,  but  to  suggest  new  elements  that  could  easily  be  incorporated  into  the 
existing  system.  These  elements  will  fill  in  the  gaps  and  mitigate  the 
vulnerabilities  that  have  already  been  recognized  in  the  current  model  and  will 
form  a  solution  that  meets  the  surveillance  objectives  being  posed. 

This  study  investigates  the  extent  to  which  the  Hellenic  Navy  will  benefit 
from  integrating  UAVs  into  the  existing  surveillance  system  in  its  AOI.  For  this 
reason,  the  sensors  that  currently  are  involved  in  this  task  are  studied  in 
compliance  with  the  complex  maritime  environment  in  which  they  operate,  like 
the  Aegean  Sea. 

E.  RESEARCH  QUESTION  AND  CORRESPONDING  HYPOTHESES 

1.  Primary  Research  Question 

Does  the  use  of  UAVs  in  a  surveillance  role  mitigate  the  limitations  of  the 
current  surveillance  system  to  detect  targets  in  complex  maritime  environments, 
assuming  that  the  target's  tactic  is  to  approach  transiting  merchant  vessels  or 
take  advantage  of  nearby  shorelines? 
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Hypotheses  for  the  Primary  Research  Question: 

•  Ho:  There  is  no  difference  in  the  surveillance  system’s  performance 
whether  it  uses  UAVs  or  not  assuming  that  the  target’s  tactic  is  to 
approach  transiting  merchant  vessels  or  to  take  advantage  of 
nearby  shorelines. 

•  Ha:  A  surveillance  system  with  UAVs  performs  better  than  one 
without  UAVs  assuming  that  the  target’s  tactic  is  to  approach 
transiting  merchant  vessels  or  to  take  advantage  of  nearby 
shorelines. 

2.  Exploratory  Research  Questions 

What  are  the  important  factors  that  contribute  to  maritime  surveillance 
when  UAVs  are  used? 

•  Is  the  number  of  transiting  neutral  vessels  an  important  factor? 

•  Is  the  length  of  the  shoreline  an  important  factor  for  the  surveillance 
system’s  performance? 

F.  OVERVIEW  OF  METHODOLOGY 

To  explore  the  problem  posed,  we  use  an  agent-based  simulation 
environment  called  Map-Aware  Non-uniform  Automata  (MANA).  In  this  simulation 
tool  we  set  up  a  model  that  represents  the  current  surveillance  system.  This 
model  is  compared,  applying  appropriate  statistical  techniques,  with  another 
which  uses  UAVs  in  addition  to  the  sensors  the  previous  model  used. 

The  input  data  are  assembled  by  reviewing  technical  data,  manufacturer 
supplied  information,  article  reviews,  and  subject  matter  expert  interviews.  Since 
the  input  values  are  not  always  fixed  or  known,  a  range  of  them  is  used.  In  this 
way,  the  uncertainties  of  the  inputs  are  mitigated,  and  some  of  the  stochastic 
effects  of  the  warfare  are  captured  (Abel,  2009).  A  Nearly  Orthogonal  Latin 
Hypercube  (NOLH)  design  is  used  to  create  a  pattern  of  different  starting 
conditions.  For  each  condition,  a  simulation  is  run  for  a  sufficient  number  of 
replications  to  estimate  the  variability  in  the  output.  Exploratory  data  analysis 
techniques  are  used  to  characterize  the  impact  of  surveillance  sensors  and 
tactics  on  the  overall  performance  of  the  surveillance  system. 
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Our  simulation  model  captures  the  following  attributes: 

•  Enemy  targets  try  to  take  advantage  of  the  presence  of  transiting 
merchant  vessels  and  the  nearby  shoreline  at  all  times. 

•  All  the  other  sensors,  besides  the  UAVs,  have  the  same  capabilities 
either  with  or  without  the  presence  of  UAV. 

•  The  assigned  UAV  covers  those  areas  that  the  ground  radars  are 
incapable  of  covering. 

The  assumptions  of  the  models  we  create  are  presented  in  detail  in 
Chapter  III. 

G.  BENEFITS  OF  THE  STUDY 

This  study  will  benefit  the  decision  makers,  either  officers  or  politicians, 
who  need  to  understand  how  UAVs  might  be  force  multipliers.  In  other  words,  our 
research  will  test  the  value  of  UAVs  in  monitoring  complex  maritime 
environments  and  provide  the  initial  guidelines  for  the  use  of  UAVs  in  the 
maritime  domain.  Additionally,  results  from  this  study  will  assist  decision  makers 
in  forming  the  future  policy  and/or  tactics,  techniques  and  procedures  for  small 
friendly  warships  to  counteract  UAVs. 

We  are  expecting  that  the  results  of  this  study  will  be  applied  in  similar 
surveillance  situations  in  different  domains.  A  potential  area  of  exploration  could 
be  the  monitoring  of  maritime  traffic  in  the  vicinity  of  ports.  Another  area  could  be 
in  border  security  when  combining  ground,  aerial  and  maritime  sensors.  This 
solution  could  be  investigated  for  situations  where  potential  illegal  activities  might 
happen  using  various  techniques  to  mislead  the  detection  sensors  both  on  the 
ground  and  in  the  sea  or  rivers. 

H.  THESIS  FLOW 

Chapter  II  offers  a  description  of  the  AOI  and  of  the  concept  of 
surveillance  operations.  It  continues  with  a  discussion  of  the  illegal  activities  that 
may  be  encountered.  Information  about  the  agencies  and  units  involved  in  the 
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current  surveillance  system  in  Greece  is  also  provided.  Finally,  the  airborne 
platforms  currently  in  use  by  the  U.S.  Navy  that  might  be  fitted  in  the  case 
studied  are  described. 

Chapter  III  begins  with  an  overview  of  the  modeling  tool,  MANA,  and  the 
scenario  description.  Moreover,  it  provides  a  detailed  description  of  the  model 
and  the  way  it  is  built. 

Chapter  IV  presents  the  design  of  experiment,  as  well  as  a  description  of 
the  variables  used  in  the  analysis. 

Chapter  V  starts  with  an  overview  of  the  analytical  techniques  being  used. 
It  continues  with  the  data  obtained  from  the  simulation,  and  it  closes  with  the 
analysis  section  using  appropriate  statistical  techniques. 

Chapter  VI  concludes  this  thesis,  summarizing  the  results,  giving  a 
thorough  discussion  of  the  analysis  and  offering  follow-on  problems  and 
recommendations.  The  complete  analytical  work  is  contained  in  the  Appendices. 
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II.  BACKGROUND 


A.  AREA  OF  INTEREST 

Greece  is  an  EU  Member  State  in  South-Eastern  Europe  and  is  mostly 
surrounded  by  sea.  It  has  a  coastline  on  the  Ionian  Sea  in  the  west,  the  Adriatic 
Sea  in  the  northwest,  the  Aegean  Sea  in  the  east  and  the  Eastern  Mediterranean 
in  the  south  and  southeast.  Its  coastline  length  is  an  estimated  17,400  km.  In 
Greece’s  territory  there  exist  about  10,000  islands  and  islets  making  up  around 
70  percent  of  country’s  coastline.  Greece  has  a  6  nm  territorial  sea,  but  has 
repeatedly  declared  that  “it  reserves  its  legitimate  right  under  international  law  to 
establish  a  12  nm  territorial  sea  at  a  time  deemed  appropriate”  (European 
Commission  Study,  2013). 

Greece’s  most  significant  maritime  activities  are  maritime  transport, 
marine  and  coastal  tourism,  and  fisheries.  Greece  has  the  largest  merchant  fleet 
in  the  EU  and  one  of  the  largest  merchant  fleets  in  the  world.  The  country  is 
surrounded  by  a  rather  large  number  of  important  shipping  lanes  (Figure  3)  and 
has  20  ports  that  process  more  than  one  million  tons  of  cargo  per  year  (European 
Commission  Study,  2013). 
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Figure  3.  Major  shipping  routes  near  Greece  (From  Safemed  Project, 

www.safemedgis.org). 

The  traffic  in  the  Greek  seas  is  enhanced  by  commercial  shipping  among 
the  ports,  recreational  boating,  as  well  as  ferry  transportation.  Additionally, 
Greece  is  a  popular  tourist  destination  (ranked  15th  worldwide).  Tourism  is 
extremely  important  for  the  Greek  economy  because  it  contributes  more  than  18 
percent  to  the  annual  Gross  National  Product.  A  large  number  of  cruise  ships  sail 
the  Greek  waters,  especially  during  summer. 

Given  Greece’s  unique  geographical  features  and,  in  particular,  its 
extensive  insular  territory,  the  country’s  territorial  and  social  cohesion  depends 
directly  on  the  existence  of  frequent  and  reliable  coastal  shipping  services 
(serving  94  islands,  144  ports  and  around  36  million  passengers  per  year) 
(European  Commission  Study,  2013).  More  information  about  maritime  transport 
can  be  found  in  Table  1 . 
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Table  1 . 


Information  about  maritime  transport  (From  European  Commission 

Study,  2013). 


-  3  996  ships  in  controlled  fleet  (>1  000  GT.  ‘09) 

-  Important  container  ports  (container  throughput  in 

-  969  ships  under  Greek  flag  (>1  000  GT,  ‘09) 

TEU,  ‘07):  Piraeus:  1  373  000:  Thessaloniki: 

447000 

-  Feny  passengers  ( cabotage  ‘09):  64  648  474 

-  Important  cargo  ports  (metric  tonnes.  ‘07): 

-  Largest  merchant  fleet  in  EU  and  in  the  world: 

Piraeus:  20  122  000:  Thessaloniki:  8  466  000 

o  23.48%  of  EU  fleet  in  gt  (‘08) 

-  Seaborne  passenger  transport  between  ports  ( '07): 

o  1 3  %  of  the  world's  gross  tonnage  (Greek- 
owned  fleet.  '09) 

45  858000 

This  study  focuses  only  on  the  Aegean  Sea  area  because,  as 
demonstrated  above,  it  is  more  important  than  the  other  seas  from  the 
perspective  of  shipping  routes,  number  of  islands  and  maritime  transport.  In 
addition,  this  area  is  more  vulnerable  to  illegal  activities,  as  the  next  section 
describes. 

B.  PROTECTING  NATIONAL  SECURITY 

Greece  is  facing  a  major  challenge  to  cope  with  both  the  large  influx  of 
mixed  migratory  flows,  including  irregular  migrants,  refugees  and  asylum 
seekers,  and  the  current  economic  crisis.  PACE,  the  Council  of  Europe’s 
Parliamentary  Assembly,  adopted  Resolution  1918  in  2013  calling  for  “firm  and 
urgent  measures  [to]  tackle  the  mounting  pressure  and  tension  over  asylum  and 
irregular  migration  into  Greece,  Turkey  and  other  Mediterranean  countries” 
(European  Council,  2013).  According  to  this  Resolution,  Greece  has  become  the 
main  entry  point  for  irregular  migratory  flows  into  the  European  Union,  while 
Turkey  has  become  the  main  country  of  transit. 

According  to  statistics  provided  by  the  United  Nations  High  Commissioner 

for  Refugees,  in  2010  more  than  132,000  third-country  nationals  were  arrested  in 
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Greece,  including  53,000  in  the  Greek-Turkish  border  regions.  During  the  first  ten 
months  of  2012,  over  70,000  arrests  occurred,  including  about  32,000  at  the 
borders  of  Turkey.  People  came  from  110  different  countries,  the  majority  from 
Asia  and  the  Middle-East.  The  irregular  migration  routes  in  the  Mediterranean 
Sea,  based  on  2012  data,  are  shown  in  Figure  4. 


Figure  4.  2012  Map  of  irregular  migration  routes  in  Mediterranean  Sea  (From 

Interactive  map  on  migration,  http://www.imap-migration.org). 


The  problem  has  been  traditionally  more  severe  in  the  Eastern 
Mediterranean  than  in  other  regions,  and  this  fact  increases  the  responsibility  of 
the  adjacent  countries  to  be  more  effective  in  monitoring  their  area  of  interest. 
The  detection  of  illegal  border-crossings  in  the  Mediterranean  Sea  is  shown  in 
Figure  5.  Although  the  situation  on  the  Western  route  is  not  alarming,  on  the 
Eastern  route  detections  have  followed  a  remarkably  seasonal  pattern  over  the 
last  two  years  with  a  constantly  higher  degree  of  illegal  activity.  In  the  second 
quarter  of  2012,  there  were  14,125  detections  of  illegal  border-crossing  on  the 
Eastern  Mediterranean  route,  an  increase  of  27  percent  compared  to  the  same 
period  in  2011.  Concerning  the  Central  Mediterranean  route,  we  notice  an 
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increase  of  detections  of  illegal  border-crossings  during  201 1  due  to  the  turbulent 
sociopolitical  developments  in  North  Africa  known  as  the  Arab  Spring 
(FRONTEX,  2012). 


2008_ 2009_ 2010_ 2011_ 2012 

Figure  5.  Detections  of  illegal  border-crossing  between  BCPs  by  main  irregular 

migration  routes  (From  FRONTEX,  2012). 

Most  migrants  and  asylum  seekers  use  the  Greek  area  as  an  intermediate 
step  towards  Europe.  Many  of  them  cannot  exit  the  country  due  to  border  checks 
and  arrests,  and  this  restriction  is  one  reason  many  of  them  return  to  their 
country  of  origin.  The  serious  economic  crisis  of  Greece  aggravates  the  situation 
and  negatively  affects  the  efforts  of  the  Greek  authorities  to  respond  to  the  large 
influx  adequately  (Strik,  2013). 

The  Greek  Government  has  taken  a  number  of  measures  to  address  the 
problem  involving  the  land  borders  between  Greece  and  Turkey,  and  these 
efforts  have  resulted  in  a  tremendous  decrease  of  irregular  land  border  crossing. 
However,  this  fact  impelled  illegal  migrants  to  use  of  sea  routes  between  the  two 
countries.  During  the  last  months,  many  reports  show  that  migrants  have  been 
arriving  on  Greek  islands  of  the  Central  and  South  Aegean  Sea.  Between  August 

and  December  2012,  3,280  persons  were  arrested  after  crossing  the  Greek- 
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Turkish  sea  border  as  compared  to  65  persons  in  the  first  seven  months  of  2012. 
Moreover,  an  increase  of  deaths  at  sea  has  been  confirmed.  In  September  2012, 
60  people  perished  when  their  boat  sank  off  the  coast  in  Izmir,  whereas  on  15 
December  2012,  20  bodies  were  found  at  sea  off  the  coast  of  Lesvos  (Reuters, 
2012). 

Not  all  of  these  migrants  are  individuals  seeking  asylum.  Many  cases  of 
irregular  border-crossing  have  involved  human  trafficking.  In  addition,  smugglers 
might  also  be  involved  in  cases  of  drugs  and  weapons  trafficking,  especially  after 
the  serious  instability  in  the  area  since  2012,  caused  by  the  political  problems  in 
Syria.  Surveillance  capabilities  should  be  enhanced  not  only  to  curtail  these 
illegal  activities  but  also  to  improve  the  legal  fishery  enforcement  and  the  reaction 
time  in  Search-and-Rescue  (SAR)  operations. 

C.  CONCEPT  OF  OPERATIONS 

As  stated  in  the  previous  chapter,  there  are  a  lot  of  different  sensors  that 
contribute  to  the  RMP  which  is  defined  as  “a  composite  picture  of  activity  of  a 
maritime  area  of  interest  for  a  given  time”  (Dore  et  al.,  2002).  To  build  a  coherent 
RMP,  a  large  number  of  civilian,  military,  and  allied  sensor  systems  must 
collaborate  in  a  systematic  way  to  gather  as  much  information  as  is  possible 
about  the  activities  of  the  targets  existing  in  the  AOI.  This  information  will  be  used 
by  the  decision  makers  at  the  operational  and  tactical  level  to  plan  and  assign 
their  assets  accordingly. 

The  RMP  helps  to  provide  commanders  with  a  Common  Operational 
Picture  (COP).  The  Joint  Publication  (JP)  3-0  defines  the  COP  as  “a  single 
identical  display  of  relevant  information  shared  by  more  than  one  command  that 
facilitates  collaborative  planning  and  assists  all  echelons  to  achieve  situational 
awareness.”  In  other  words,  RMP  assists  the  commanders  in  obtaining  the 
intelligence  needed  to  use  their  assets  more  effectively  and,  consequently, 
achieve  their  mission.  The  purpose  of  the  COP  is  to  provide  real-time  information 
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on  Blue,  Red,  and  White  forces  (friendly,  enemy,  and  neutral  vessels)  operating 
in  the  maritime  domain. 

Finally,  the  goal  is  to  improve  the  Maritime  Domain  Awareness  (MDA), 
which  is  defined  in  the  JP  3-32  as  the  “effective  understanding  of  anything 
associated  with  the  maritime  domain  that  could  impact  the  security,  safety, 
economy,  or  environment  of  a  nation.”  To  achieve  complete  MDA,  all  mission  or 
functional  areas  of  the  components  and  agencies  tasked  must  be  incorporated 
into  a  common  architecture  that  provides  each  with  the  ability  to  share  near  real¬ 
time  information,  synthesize  inputs  from  multiple  sources,  and  quickly  analyze 
the  data  to  affect  improved  decision  making  before  the  opportunity  to  investigate 
and  act  on  identified  threats  is  lost. 

To  achieve  the  desired  detection  of  enemy  combatants,  the  intelligence 
community  utilizes  an  assortment  of  Intelligence,  Surveillance,  and 
Reconnaissance  (ISR)  sensors.  According  to  JP  1-02,  ISR  is  “an  activity  that 
synchronizes  and  integrates  the  planning  and  operation  of  sensors  and  assets, 
as  well  as  the  processing,  exploitation,  and  dissemination  of  information  in  direct 
support  of  current  and  future  operations.”  The  JP  1-02  further  defines  the  three 
ISR  components  individually: 

•  Intelligence — the  product  resulting  from  the  collection,  processing, 
integration,  evaluation,  analysis,  and  interpretation  of  available 
information  concerning  foreign  nations,  hostile  or  potentially  hostile 
forces  or  elements,  or  areas  of  actual  or  potential  operations. 

•  Surveillance — the  systematic  observation  of  aerospace,  surface,  or 
subsurface  areas,  places,  persons,  or  things,  by  visual,  aural, 
electronic,  photographic,  or  other  means. 

•  Reconnaissance — a  mission  undertaken  to  obtain,  by  visual 
observation  or  other  detection  methods,  information  about  the 
activities  and  resources  of  an  enemy  or  adversary. 

Intelligence  is  broader  term,  while  surveillance  refers  to  systematic  and 
constant  observation  of  an  area  or  a  target,  and  reconnaissance  refers  to 
activities  performed  to  obtain  all  the  information  needed  about  a  detected  contact 
(Brown  &  Schulz,  2009). 
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D.  AGENCIES  INVOLVED  IN  CURRENT  SURVEILLANCE  SYSTEM 

In  this  section,  the  agencies  that  currently  participate  in  the  Surveillance 
System  of  the  Aegean  Sea  are  briefly  presented.  The  Surveillance  System  of  the 
Aegean  Sea  consists  of  sensors  belonging  to  different  organizations.  Because  of 
the  number  of  organizations  and  sensors  involved,  it  is  challenging  to  achieve 
completely  accurate  and  timely  information.  The  stakeholders  that  currently 
contribute  to  the  National  Surveillance  System  include  the  following: 

1.  Hellenic  Navy 

Hellenic  Navy  (HN)  is  the  main  surveillance  authority  in  the  Maritime 
Greek  territory.  HN  deploys  its  units  taking  into  consideration  the  information 
from  the  other  users  (discussed  later)  and  the  Operational  Plan  the  leaders  draw. 
The  ships  (including  frigates,  fast  patrol  boats  and  submarines)  are  equipped 
with  the  newest  devices  with  impressive  capabilities.  Although  various  types  of 
ships  might  be  involved  in  ISR  Operations,  a  typical  asset  is  the  Gun  Boat 
Osprey  HSY  56A  (Figure  6). 


Figure  6.  HS  AITTITOS  (P-  268)  (From  Hellenic  Navy,  www.hellenicnavy.gr). 
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The  navigation  radar,  the  tracking  radar  and  the  infrared  cameras  provide 
advanced  detection  capabilities,  while  the  radar’s  increased  range  capability  is 
an  advantage  when  there  is  need  for  covering  large  areas.  The  combination  of  its 
small  draft  with  its  relatively  large  displacement  makes  this  gun  boat  capable  of 
operating  in  shallow  waters  and  in  adverse  weather  conditions.  The  drawback  is 
the  Osprey’s  weakness  in  achieving  high  speed  (>35  kn),  which  makes  it 
incapable  of  pursuing  fast  targets.  The  general  characteristics  of  this  type  of  ship 
are  depicted  in  Table  2. 

Table  2.  General  characteristics  of  HS  MACHITIS  (Type  Osprey  HSY  56A). 


Length  Overall 

56.5  m 

Beam  (max.) 

10  m 

Draft 

3.6  m 

Displacement 

575  tons 

Speed 

22  knots 

Range 

2,000  miles 

Engines 

2  Wartsila  Diesel  16  V  25  2  X  5000 
HP 

Weapons 

1  OTOMELARA  3"  (76mm)/62 

1  OTOBREDA  40mm/70 

2  RHEIMENTALL  20mm 

1  STINGER 

Devices/Systems 

□  ROD  MK  2  /  TV  CAMERA  (FC) 
MIRADOR  TVT-IRT  Camera,  Laser 
Firing  Mode 

VARIANT  Air-Surface  Radar 
BridgeMaster  E  (Decca)  NAVRAD 
RL80C  MARPA  (Raytheon) 

NAVRAD 

ESM  /  DR  3000  SLW 

TACTICOS  (SMCS)  /  LINK  1 1 

Additionally,  the  HN  controls  a  number  of  ground  radars  located  on 

several  islands  in  Central  and  Eastern  Aegean  Sea,  operated  by  military 

personnel.  These  radars  collaborate  with  other  sensors  (i.e. ,  radars,  warships, 

merchant  ships,  etc.)  in  the  adjacent  areas  to  create  the  RMP.  Although  they 

achieve  increased  detection  ranges,  these  radar  systems  do  not  have  a  360- 
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degree  Field  of  View  (FOV)  due  to  various  physical  obstacles  (i.e.,  higher  hills, 
mountains)  resulting  in  the  creation  of  “blind  sectors.”  The  ground  radar  system’s 
location  is  chosen  taking  into  consideration  a  number  of  factors,  such  as  what 
area  is  more  important  to  cover,  what  elevation  provides  higher  detection  range, 
what  locations  are  capable  of  accommodating  these  stations  and  others. 

Additionally,  until  recently,  the  HN  operated  a  fleet  of  Maritime  Patrol 
Aircraft  (MPA)  Orion  P-3B,  but  due  to  economic  deficiencies,  they  are  not 
currently  active.  The  MPA’s  missions  were  mostly  in  surveillance  and  anti¬ 
submarine  warfare.  These  units  were  capable  of  flying  at  9,000m,  and  their 
operational  endurance  was  about  10  hours. 

The  HN  also  has  the  responsibility  of  exchanging  information  with 
numerous  International  stakeholders  (NATO,  Partnership  for  Peace  countries 
and  the  EU)  as  will  be  explained  later  in  this  chapter. 

2.  Hellenic  Coast  Guard 

The  Hellenic  Coast  Guard  is  a  paramilitary  organization  that  can  support 
the  Hellenic  Navy  in  wartime,  but  it  resides  under  the  control  of  the  Ministry  of 
Merchant  Marine  and  Aegean  Sea  in  times  of  peace.  The  force  has  158  patrol 
craft  of  various  types  and  four  light  aircraft.  The  Coast  Guard  patrols  all  Greek 
harbors,  coastlines,  and  territorial  waters,  monitoring  anti-pollution  measures  and 
controlling  merchant  shipping.  The  Hellenic  Coast  Guard  has  permanently 
deployed  small  patrol  ships  in  the  major  islands  of  the  Central  and  Eastern 
Aegean  Sea,  and  their  task  is  mostly  to  deter  smuggling  and  illegal  immigration, 
and  to  perform  SAR  operations.  A  typical  Coast  Guard  vessel  that  is  used  to 
patrol  is  the  Panther  57  (Lambro57)  (Figure  7),  which  is  capable  of  achieving 
speeds  over  40  kn.  Its  shallow  draft  and  protective  water  jet  intake  grill  make  it 
capable  of  operating  in  waters  as  shallow  as  one  meter  deep.  In  contrast,  it  is 
equipped  with  only  navigation  radar  and  night  vision  binoculars,  and  due  to  small 
tonnage,  it  is  extremely  vulnerable  to  weather  conditions.  The  general 
characteristics  of  this  type  of  ship  are  depicted  in  Table  3. 
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Figure  7.  Hellenic  Coast  Guard’s  typical  patrol  boat  (Panther  57)  (From 
MotoMarine  shipyards,  www.motomarine.gr). 


Table  3.  General  characteristics  of  Panther  57 


Length  Overall 

18.2  m 

Beam  (max.) 

4.68  m 

Draft 

0.92  m 

Displacement 

28  tons 

Max  Speed 

44  knots 

Range 

250  miles 

Engines 

2  Engine  Type  12V  Man 

Weapons 

0,50  Browning  Automatic  Cannon 

3.  Hellenic  Air  Force 

Surface  Surveillance  is  not  among  the  tasks  of  the  Hellenic  Air  Force. 
Nevertheless,  use  of  RF-4  aircraft  can  contribute  to  reconnaissance,  especially 
during  wartime  or  SAR  operations  in  peacetime. 

4.  Hellenic  Army 

The  large  number  of  islands  in  the  Aegean  Sea,  either  big  habitable  ones 
or  small  islets,  forms  a  rare  environment  for  operations.  Being  on  an  island  one 
may  have  a  view  of  the  neighboring  island(s)  most  of  the  time,  and  this  allows  a 
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person  to  obtain  a  clear  understanding  of  the  interim  sea  space.  Trying  to  take 
advantage  of  this  spatial  uniqueness,  the  Hellenic  Army  has  located  a  number  of 
coastal  (sea)  and  Ground  Surveillance  Radar  BOR-A550  (Figure  8)  suitable  for 
army,  border/coast  guard  and  security  applications.  It  combines  surveillance  of 
ground,  sea  and  lower  level  airspace  in  a  single  radar  system  whereby  moving 
targets  will  be  detected,  located,  automatically  classified  and  tracked  day  and 
night  under  severe  weather  conditions. 

Furthermore,  across  the  Greek  coastline  of  the  Eastern  Aegean  Sea 
islands,  there  are  a  large  number  of  sites  with  observers  equipped  with  basic 
vision  tools  (i.e. ,  binoculars,  night  vision  goggles).  All  the  information  about 
targets  in  the  AOI  is  passed  to  the  National  Surveillance  System  so  that  all  the 
stakeholders  are  aware  of  the  RMP. 


Figure  8.  Radar  BOR-A  550  (From  Hellenic  Army,  www.army.gr). 


5.  NATO 

The  Maritime  Command  and  Control  Information  System  (MCCIS)  is  a 
military  maritime  command  and  control  system  that  has  been  developed  and 
maintained  for  members  of  NATO.  The  MCCIS  assists  strategic  and  tactical 
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commanders  and  their  staffs  in  the  decision  making  process.  MCCIS  is  installed 
and  in  operational  use  at  61  sites  in  different  countries,  one  of  which  is  Greece. 
There  are  over  250  users.  All  the  member  states  are  encouraged  to  contribute  to 
the  RMP,  and  they  also  have  the  privilege  of  sharing  all  the  data  without 
censorship. 

MCCIS  electronically  processes  data  from  multiple  sources,  displays  data 
in  various  command  and  control  applications,  and  allows  the  user  to  manipulate 
this  data.  One  important  component  in  NATO’s  MDA  is  NATO’s  Maritime  Safety 
and  Security  Information  System  (MSSIS),  which  is  based  around  the  acquisition 
and  analysis  of  Automatic  Identification  System  (AIS)  data.  This  data  is  gathered 
from  the  AIS  systems  of  NATO  member  states  as  well  as  by  a  number  of  other 
non-NATO  states  on  the  basis  of  bilateral  agreements. 

Once  the  MSSIS  analysis  is  complete,  the  relevant  data  is  fed  into 
NATO’s  MCCIS,  as  shown  in  Figure  9.  Data  held  within  the  MCCIS  is  classified 
as,  in  addition  to  containing  the  compiled  picture  created  through  MSSIS,  it  also 
includes  intelligence  data,  classified  surveillance  data  (from  satellites  and  other 
sensors)  and  the  real  time  location  of  NATO  assets  (such  as  warships).  One  of 
the  reasons  why  the  basic  MSSIS  is  kept  at  an  unclassified  level,  however,  is  to 
attract  the  participation  of  non-NATO  states  (European  Commission,  2008). 

As  regards  the  acquisition  and  processing  of  surveillance  data,  maritime 
or  otherwise,  NATO  does  not  have  an  explicit  mandate  as  such.  Instead  such 
activities  are  to  be  implied  from  the  content  of  Article  3  of  the  North  Atlantic 
Treaty,  which  states:  “In  order  more  effectively  to  achieve  the  objectives  of  this 
Treaty,  the  Parties,  separately  and  jointly,  by  means  of  continuous  and  effective 
self-help  and  mutual  aid,  will  maintain  and  develop  their  individual  and  collective 
capacity  to  resist  armed  attack.” 
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Figure  9.  Diagram  of  MSSIS  (From  European  Commission,  2008). 


6.  European  Union 

In  2004,  the  European  Agency  for  the  Management  of  Operational 
Cooperation  at  the  External  Borders  of  the  Member  States  of  the  European 
Union  (FRONTEX)  was  established,  and  its  Eastern  Sea  Borders  Centre  is 
located  in  Piraeus,  Greece.  FRONTEX’s  mission  is  to  help  EU  member  states 
implement  EU  rules  on  external  border  controls  and  to  coordinate  operational 
cooperation  between  member  states  in  the  field  of  external  border  management. 

While  it  remains  the  task  of  each  member  state  to  control  its  own  borders, 
the  agency  is  vested  with  the  function  of  ensuring  that  they  all  do  so  with  the 
same  high  standard  of  efficiency.  To  reduce  duplicated  effort  and  hence  save 
time,  money  and  other  resources,  the  European  Patrols  Network  was  born  in 
2007.  Greece,  as  guardian  of  the  Eastern  borders,  has  been  assisted  periodically 
by  other  EU  countries  with  the  deployment  of  a  few  patrol  ships  and  aircraft. 

The  EU  also  has  developed  SafeSeaNet,  a  vessel  traffic  monitoring  and 
information  system,  established  to  enhance  maritime  safety,  port  and  maritime 
security,  marine  environment  protection  and  efficiency  of  maritime  traffic  and 

26 


maritime  transport.  It  was  established  as  a  centralized  European  platform  to 
facilitate  the  maritime  information  sharing  between  European  countries.  The  main 
sources  of  information  consist  of  the  AIS  reports  and  notification  messages  sent 
by  designated  authorities  in  participating  countries. 

On  February  13,  2008  the  European  Commission  adopted  a 

Communication  on  the  creation  of  a  European  Border  Surveillance  System.  It 
was  designed  to  support  the  member  states  in  their  efforts  to  reduce  the  number 
of  illegal  immigrants  entering  the  European  Union  by  improving  their  situational 
awareness  at  their  external  borders  and  increasing  the  reaction  capability  of  their 
information  and  border  control  authorities  (Commission  of  the  European 
Communities,  2008).  It  also  focuses  on  the  EU's  southern  and  eastern  maritime 
borders. 

E.  AIRBORNE  PLATFORMS 

Unmanned  aircraft  are  commonly  called  unmanned  aerial  vehicles 
(UAVs),  and  when  combined  with  ground  control  stations  and  data  links,  they 
form  a  UAS,  or  unmanned  aerial  system.  Conventional  wisdom  states  that  the 
UAS  offers  two  main  advantages  over  manned  aircraft:  they  are  considered  more 
cost-effective,  and  they  minimize  the  risk  to  a  pilot’s  life  (U.S.  Library  of 
Congress,  201 1).  Because  this  study  attempts  to  investigate  the  ways  in  which  a 
UAS  can  be  integrated  into  the  current  surveillance  system,  its  basic  applications 
and  the  major  systems  which  might  be  used  are  being  discussed  in  this  section. 

JP  3-52  defines  a  UAS  as  “that  system  whose  components  include  the 
necessary  equipment,  network,  and  personnel  to  control  an  unmanned  aircraft.” 
According  to  this  publication,  the  term  UAS  will  be  used  instead  of  UAV  across 
the  U.S.  Department  of  Defense.  In  this  study  the  UAS  Program  of  U.S.  Navy  is 
used  as  a  reference,  but  the  terms  UAVs  and  UAS  are  used  interchangeably. 
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1.  Applications  for  UAVs 

From  the  U.S.  Navy’s  perspective,  UAVs  are  currently  being  used  in  a 
large  variety  of  applications.  Of  course  not  every  UAV  is  capable  of 
accomplishing  all  the  missions,  but  each  type  of  UAV  is  designed  to  carry 
different  payloads  and  sensors;  for  this  reason,  UAVs  can  be  used  for  a  variety  of 
applications.  Some  of  the  applications  that  the  current  UAVs  are  used  for  or  for 
which  they  are  about  to  be  used  in  the  near  future  are:  penetrating  strike,  ISR 
operations,  ELINT  (intelligence  derived  from  electromagnetic  radiations  from 
foreign  sources)  Collection,  COMINT  (technical  and  intelligence  information 
derived  from  foreign  communications)  Collection,  air-to-air  combat,  airborne 
electronic  attack,  suppression  of  enemy  air  defenses,  close  air  support, 
chemical-biological-radiological-nuclear  detection,  battle  damage  assessments 
(BDA),  mine  detection,  precision  target  designation,  anti-submarine  operations, 
suppression  of  enemy  air  defenses  (SEAD)  and  psychological  operations,  such 
as  dropping  leaflets  (Alkire,  2010;  U.S.  Library  of  Congress,  2011). 

In  a  2004  study,  each  Combatant  Command  and  Service  was  asked  to 
rank  the  importance  of  18  missions  relative  to  four  general  classes  of  UAS  (i.e. , 
small,  tactical,  theater,  and  combat).  Their  responses  are  depicted  in  Table  4. 
Although  the  distinction  between  the  classes  of  UAS  is  not  the  same  across  time, 
it  is  notable  that  Reconnaissance,  what  we  are  most  interested  in  for  this  study, 
is  ranked  as  the  highest  priority  regardless  of  the  type  of  platform. 


28 


Table  4. 


Combatant  commander/service  UAS  mission  prioritization  matrix 
(From  Office  of  the  Secretary  of  Defense,  2005). 


Mission 

Small 

Tactical 

Theater 

Combat 

JFC 

Reconnaissance 

• 

d 

d 

d 

BA 

Signals  Intel 

10 

3 

2 

5 

BA 

Mine  Detection/CM 

7 

11 

13 

14 

FP 

Precision  Target  Location  and 
Designation 

2 

2 

3 

2 

FA 

Battle  Management 

4 

10 

4 

7 

C2 

Chem/Bio  Reconnaissance 

3 

7 

6 

9 

BA 

Counter  Cam/Con/Deception 

8 

5 

7 

11 

BA 

Electronic  Warfare 

14 

9 

10 

4 

FP 

Combat  SAR 

6 

8 

8 

10 

FA 

Communications'Data  Relay 

5 

6 

5 

8 

C2 

Information  Warfare 

15 

12 

n 

6 

FA 

Digital  Mapping 

11 

13 

9 

12 

BA 

Littoral  Undersea  Warfare 

17 

15 

14 

13 

FA 

SOF  Team  Resupply 

9 

16 

17 

16 

FL 

Weaponization'Strike 

16 

4 

12 

3 

FA 

GPS  Psuedolite 

18 

18 

15 

18 

C2 

Covert  Sensor  Insertion 

12 

14 

16 

15 

BA 

Decoy/Pathfinder 

13 

17 

18 

17 

FA 

Additionally,  UAVs  can  provide  critical  support  for  SAR  operations. 
However,  to  achieve  their  full  potential,  all  the  factors  that  affect  the  UAV’s 
performance,  such  as  payload  capabilities,  endurance,  communications  and 
environmental  hazards,  should  be  taken  into  consideration  (Waharte  &  Trigoni, 
1997). 

Moreover,  in  this  paper  we  are  more  interested  in  tasks  that  UASs  can 
assume  in  the  current  framework  of  the  HN.  Therefore,  in  this  section,  traditional 
navy  missions  are  emphasized  rather  than  naval  contributions  to  irregular 
warfare. 
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2.  Types  of  UAVs 

UAVs  can  be  divided  into  many  categories  taking  into  consideration 
various  factors  like  the  missions  they  accomplish,  their  characteristics,  the 
platform  from  which  they  are  launched  or  their  cost.  According  to  the  National 
Research  Council  (2005),  the  Navy  views  its  future  use  of  unmanned  aerial 
vehicles  to  be  primarily  in  three  categories: 

•  Long-dwell,  standoff  intelligence,  surveillance,  and  reconnaissance 
(ISR),  as  exemplified  by  the  Broad  Area  Maritime  Surveillance 
(BAMS)  concept  and  the  Global  Hawk  Maritime  Demonstration 
(GHMD); 

•  Carrier-based,  penetrating  surveillance  and  SEAD/strike  Joint 
Unmanned  Combat  Air  System  ;  and 

•  Ship-based  tactical  surveillance  and  targeting,  which  call  for  a 
vertical  takeoff-  and-landing  system  that  can  operate  from  a  variety 
of  ship  types. 

As  stated  earlier,  the  U.  S.  Navy  is  making  large  investments  in  a  number 
of  major  UAV  programs,  including  BAMS,  the  Unmanned  Combat  Aircraft 
System  Demonstrator,  the  Fire  Scout  vertical  takeoff/landing  tactical  UAS 
(VTUAS),  and  the  Small  Tactical/Tier  II  UAS  (STUAS/Tier  II  UAS)  (Alkire,  2010). 
In  general,  UAVs  can  be  launched  from  several  platforms  like  shore  bases  or 
infrastructures,  from  surface  ships  or  from  aircraft  carriers.  Because  the  latter  has 
no  implementation  in  the  HN,  these  vehicles  are  not  studied  in  this  paper.  In  the 
following  section,  we  separate  UASs  into  those  categories  that  are  of  relevance 
to  the  HN  and  briefly  present  their  main  characteristics. 

a.  Broad  Area  Maritime  Surveillance  (BAMS) 

The  U.S.  Navy  is  developing  the  BAMS  unmanned  system  to 
provide  a  persistent,  maritime,  worldwide  access,  ISR  capability  (Figure  10).  In 
FY  2003,  the  U.S.  Navy  purchased  two  RQ-4A  Global  Hawk  variants  with 
electro-optical/infrared  (EO/IR)  and  SAR  sensors,  along  with  their  ground  control 
stations  and  support  equipment.  These  are  known  as  the  GHMD  System.  It  is  a 
high-altitude,  long-endurance  (HALE)  Unmanned  Aerial  Surveillance  System  and 
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provides  the  U.S.  Navy  with  demonstration  capability  primarily  for  doctrine, 
concept  of  operations,  and  tactics,  techniques,  and  procedures  development. 
The  BAMS  UAS,  named  RQ-4N,  is  a  maritime  derivative  of  the  Global  Hawk 
equipped  with  Navy-specific  control  stations  called  Tactical  Control  Systems.  It 
will  have  a  full  360-degree  field  of  regard  and  the  capability  to  collect  full  motion 
video.  It  also  requires  a  runway  for  takeoff  and  landing  and  is  not  carrier-capable 
(Alkire,  2010). 
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Figure  10.  Graphical  representation  of  BAMS  Operations  (From  Northrop 

Grumman,  www.northropgrumman.com). 

The  aircraft  has  a  projected  12,000  nm  range  and  35-hour 
endurance,  with  satellite  and  line-of-sight  communication  links  to  the  ground 
system.  High  resolution  sensors  that  can  look  through  adverse  weather,  day  or 
night,  from  an  altitude  of  65,000  feet,  can  conduct  limited  maritime  surveillance 
over  an  area  the  size  of  the  state  of  Illinois  in  only  24  hours  (Northrop  Grumman, 
2007).  Detailed  specifications  are  shown  in  Table  5. 
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Table  5.  Specifications  of  RQ-4  Global  Hawk  Maritime  Demonstration 
System  (From  Northrop  Grumman,  www.northropgrumman.com). 


RQ-4  Air  Vehicle 

Wingspan 

116.2  ft  (35.4  m) 

Height 

14.6  ft  (4.2  m) 

Length 

44.4  ft  (13.5  m) 

Gross 

Takeoff 

Weight 

26,750  lbs 
(12,133.6  kg) 

Payload 

2,000  lbs  (907.2  kg) 

Maximum 

Altitude 

65,000  ft  (19.8  km) 

Ferry  Range 

10,000  nm 
(18.520  km) 

Loiter 

Velocity 

343  knots 

On-Station 
Endurance  at 
1,200  nm 

24  Hours 

Maximum 

Endurance 

35  Hours 

Sensors 

Synthetic 
Aperture  Radar 
(SAR) 

1 .0/0.3  M  Resolution 
(WAS/Spot) 

Maritime 

Target 

Acquisition 

Cue  for  ISAR 

Maritime 

Search 

15,000  sq.  km/Min 

ISAR 

3  Resolutions 

Electro-Optical 

NIIRS  6. 0/6. 5 
(WAS/Spot) 

Infrared 

NIIRS  5. 0/5. 5 
(WAS/Spot) 

b.  Vertical  Takeoff/  Landing  Tactical  Unmanned  Aerial 
Systems 

The  Fire  Scout  VTUAV  system  provides  remarkable  situation 
awareness  and  precision  targeting  support.  The  MQ-8B  Fire  Scout  (Figure  11) 
has  the  ability  to  take  off  and  land  on  any  aviation-capable  warship  autonomously 
and  at  appropriate  land  areas  without  any  special  infrastructure. 
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Figure  1 1 .  MQ-8B  Fire  Scout  (From  America’s  Navy,  www.navy.mil). 

The  Fire  Scout  VTUAS  has  an  operational  footprint  that  is  a  fraction 
of  that  of  the  multipurpose  MH-60-class  helicopters;  it  can  operate  from,  and 
provide  the  UAS  advantages  to,  surface  ship  platforms.  The  U.S.  Navy  has 
decided  to  replace  helicopters  with  VTUAVs  during  the  evolution  of  the  Littoral 
Combat  Ship  program  (Burgess,  2004).  This  type  of  UAV  has  been  long  tested 
and  used  by  the  U.S.  Armed  Forces.  In  2009,  110  ship  take-offs  and  landings 
were  conducted,  and  during  this  time  period  the  Fire  Scout  completed  successful 
deployments  in  real  combat  fields  (Jacobson,  2010). 

The  Fire  Scout  has  an  operating  ceiling  of  20,000  ft  and  a  total 
endurance  of  over  eight  hours  that  provides  more  than  six  hours  on  station  with  a 
standard  payload  at  1 10  nm  (200  km)  from  the  launch  site.  A  system  of  two  Fire 
Scouts  can  provide  continuous  coverage  at  110  nm.  Utilizing  a  payload  that 
includes  EO/IR  sensor  with  laser  rangefinder/illuminator  and  maritime  radar,  the 
Fire  Scout  can  find  and  identify  tactical  targets,  track  and  illuminate  them  (Figure 
12)  and  accurately  provide  targeting  data  to  strike  platforms  (Northrop  Grumman, 
2012).  Detailed  characteristics  are  shown  in  Table  6. 
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Figure  12.  Images  taken  by  MQ-8B  Fire  Scout’s  EO/IR  sensors  (From  Northrop 

Grumman,  www.NorthropGrumman.com). 


Table  6.  Specifications  of  MQ-8B  Fire  Scout  (From  Northrop  Grumman, 

www.NorthropGrumman.com). 


Fuselage  Length 
(with  Dual  Payload  Nose) 

23.95  ft  (7.3  m) 

Fuselage  Width 

6.20  ft  (1.9  m) 

Length 

(with  Blades  Folded  Forward) 

30.03  ft  (9.2  m) 

Rotor  Diameter 

27.50  ft  (8.4  m) 

Height  (Top  of  Tail  Antenna) 

9.71ft  (2.9  m) 

Gross  Weight 

3,150  lbs  (1428.8  kg) 

Engine 

Rolls  Royce  250-C20W 

Turboshaft  Engine 

Speed 

115+  Knots 

Ceiling 

20,000  ft  (6.1  km) 

Total  Flight  Time  with  Baseline  Payload 

8+  Hrs 

Total  Flight  Time  with  EO/IR  +  Radar 

7+  Hrs 

Total  Flight  Time  with  Maximum  Payload 

5+  Hrs 

Payloads 

EO/IR/LRF/Mine 

Detector/Comm.  Relay/  Maritime 
Radar 
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The  MQ-8B  Fire  Scout  VTUAV  is  used  to  recapitalize  the  capability 
of  the  aging  fleet  of  P-3  Orion  aircraft  and  provide  maritime  domain  awareness 
for  the  U.S.  Navy  (Alkire,  2010).  The  Fire  Scout  can  be  used  in  various  missions, 
such  as  SSC,  Birddog/tattletale  operations,  Maritime  Interdiction  Operations, 
BDA,  ISR  operations  and  Target  Acquisition/Anti-Submarine  Warfare,  border 
patrol,  SAR  operations  and  medical  resupply  (Berner,  2004). 

c.  Small  Tactical/Tier  II  Unmanned  Aerial  Systems  (STUAS) 

In  2005,  the  U.S.  Navy  signed  a  $14.5  million  contract  with  Boeing 
to  provide  ISR  coverage  with  the  Scan  Eagle  Small  Tactical  UAS  (STUAS) 
(Alkire,  2010).  In  2009,  a  Scan  Eagle  UAS  was  operated  from  a  nearby  U.S. 
Navy  vessel  to  provide  real-time  situational  awareness  during  a  Somali  pirate 
incident  that  ultimately  ended  with  the  safe  release  of  the  captain  of  a  U.S.  cargo 
ship.  Following  the  successful  tests  and  in-the-field  support,  in  July  2010  the 
Department  of  the  Navy  awarded  Insitu  a  two-year,  $43.7  million  contract  for  the 
design,  development,  integration  and  test  of  the  STUAS  Integrator  RQ-21A  for 
use  by  the  Navy  and  Marine  Corps  (U.S.  Library  of  Congress,  2012). 

For  the  U.S.  Navy,  the  RQ-21A  (Figure  13)  will  provide  persistent 
Reconnaissance,  Surveillance,  and  Target  Acquisition  support  for  tactical 
maneuver  decisions  and  unit-level  force  defense/force  protection  for  Navy  ships, 
Marine  Corps  land  forces,  Navy  Expeditionary  Combat  Command  forces  and 
Navy  Special  Warfare  Units  (Small  tactical  unmanned  aircraft  system,  n.d.). 
According  to  the  manufacturer,  upon  completion  of  the  project  the  RQ-21 A  will  be 
able  to  handle  the  following  missions:  Search  and  Rescue,  Disaster  Response, 
Force  Protection,  Combined  Arms,  Target  Following,  Battle  Damage 
Assessment,  Pattern  of  Life,  Border  Security,  Asset  Protection,  Wildlife 
Monitoring,  Agricultural  Assessment,  Communications  Relay,  Networked 
Operations  and  Anti-Piracy  (Insitu,  2012). 
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Figure  13.  The  Integrator  RQ-21A  is  recovered  with  Insitu's  SkyHook  capture  rope 
after  its  first  operational  take  offs  and  landings  at  sea  from  the  San 
Antonio  class  dock  landing  ship  USS  Mesa  Verde  (LPD-19)  in  the  Gulf 
of  Mexico  on  10  February  2013  (From  Naval  Drones, 
www.navaldrones.com). 

The  RQ-21A  will  consist  of  a  number  of  Air  Vehicles  (AVs),  Ground 
Control  Systems  (GCS)  and  multi-mission  payloads,  which  will  provide 
intelligence  coverage,  surveillance,  reconnaissance,  and  communications  relay 
for  up  to  15  hours  per  day  continuously  with  a  short  surge  capability  for  24  hours 
a  day.  Payloads  include  Day/Night  Full  Motion  Video  cameras,  infrared  marker, 
laser  range  finder  and  AIS  receivers.  Ancillary  equipment  includes 
launch/recovery  mechanisms,  tactical  communications  equipment  and  spares. 
The  RQ-21A  will  have  a  minimal  operating  radius  of  50  nm,  and  the  AV  will  be 
capable  of  airspeeds  up  to  80  nm  per  hour  (knots)  with  a  service  ceiling  of 
15,000  ft  density  altitude.  The  fully  autonomous  launch  and  recovery  system  will 
require  minimal  space  for  takeoff  and  recovery  from  an  unimproved 
expeditionary/urban  environment,  as  well  as  from  the  deck  of  Navy  ships.  Initial 
operational  capability  is  planned  for  2013  (Small  tactical  unmanned  aircraft 
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system,  n.d.).  Characteristics  of  UAS  Integrator  (RQ-21A)  are  depicted  in 
Table  7. 

Table  7.  Specifications  of  Integrator  (RQ-21A)  (Insitu,  2012  ). 


Wingspan: 

16  ft/ 4.8  m 

Length: 

7.2  ft/ 2.2  m 

Empty  structure  weight: 

80  lb  /  34.0  kg 

Max  takeoff  weight: 

135  lb/ 61.2  kg 

Max  horizontal  speed: 

80+  knots  /  41 .2+  m/s 

Cruise  speed: 

55  knots  /  28.3  m/s 

Ceiling 

>15,000  ft/ 4,573  m 

Endurance: 

24  hours 

Payloads 

Electro-optic,  Mid-wave  infrared,  IR 
marker*,  Laser  rangefinder* 

*  Class  3B  laser  product. 

F.  DETECTION  PROBLEM  AND  RELATED  STUDIES 

This  section  briefly  discusses  the  problem  of  detection,  which  involves  the 
major  issues  that  impede  the  detection  and  identification  procedure,  as  well  as 
the  most  important  studies  made  on  surveillance  from  UAVs  either  on  the  ground 
or  in  the  maritime  domain. 

1.  Detection  Problem 

The  detection  and  identification  problem,  as  one  can  easily  understand,  is 
very  complex  as  it  involves  a  variety  of  sensors  and  agencies.  The  sensors 
involved  are  mostly  Radio  Detection  and  Ranging  (RADAR  or  radar)  systems 
and  Electro-optical/infrared  (EO/IO)  sensors.  Radar  systems  are  active  sensors 
that  emit  electromagnetic  waves  which  are  reflected  from  targets  and  then 
received  back  by  their  sensors.  Compared  to  EO/IR  systems,  radar  systems 
have  almost  the  same  performance  during  day  or  night  under  clear  weather,  but 
radar  systems  suffer  less  atmospheric  attenuation  than  EO/IR  systems.  For  this 
reason,  they  are  able  to  detect  targets  at  greater  ranges. 

Conversely,  EO/IR  systems  are  passive  devices  and  work  with  the 
radiations  emitted  by  the  target.  EO  sensors  provide  visual  images  of  the  targets 
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and  are  mostly  used  during  day.  However,  they  are  limited  by  any  factor  (such  as 
clouds,  smog  or  camouflage)  that  hampers  visual  contact  between  the  sensor 
and  the  target.  IR  cameras  transform  the  thermal  energy  that  the  targets  emit  to 
a  video  signal.  The  drawback  of  IR  sensors  is  that  they  cannot  provide  accurate 
image  details  of  the  target,  but  an  experienced  user  can  easily  determine  the 
type  of  target  from  the  resulting  image.  In  the  field,  units  with  EO/IR  sensors  are 
used  to  identify  targets  that  have  already  been  detected  by  other  sensors  (radar, 
other  reports,  intelligence,  etc.)  because  in  that  way  the  target  information  is 
received  in  less  time  (Brown  &  Schulz,  2009). 

Taking  into  consideration  that  UAVs  are  based  on  the  EO/IR  that  they 
carry,  sea  state  and  adverse  weather  conditions,  such  as  rain  droplets,  snow,  fog 
and  hail  stones,  will  all  affect  their  operation.  Moreover,  limited  endurance  or 
incapability  to  operate  in  the  aforementioned  weather  conditions  may  influence 
the  UAV’s  performance.  These  conditions  may  affect  the  unmanned  vehicle  itself 
or  the  platform  from  which  it  is  launched  (Johnson,  2004). 

Perhaps  the  most  important  factor  that  affects  the  IR  performance  is  rain. 
Precipitation  causes  attenuation  of  the  signal  (Figure  14)  and  also  could  result  in 
a  loss  of  communication  between  the  GCS  and  the  UAV  because  radio 
frequencies  are  often  used  for  the  communication  channel.  The  impact  of  surface 
clutter  diminishes  EO/IR  sensor  effectiveness,  as  well  (Johnson,  2004). 
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Figure  14.  Specific  attenuation  for  rain  calculated  for  two  model  drop  size 

distributions  (From  Crane,  1981). 

The  target  resolution  of  a  radar  system  has  the  ability  to  distinguish 
between  targets  that  are  very  close  in  either  range  or  bearing.  Weapons-control 
radar,  which  requires  great  precision,  should  be  able  to  distinguish  between 
targets  that  are  only  yards  apart.  Search  radar  is  usually  less  precise  and  only 
distinguishes  between  targets  that  are  hundreds  of  yards  or  even  miles  apart. 
Resolution  is  usually  divided  into  two  categories:  range  resolution  and  bearing 
resolution. 

Range  resolution  is  the  ability  of  a  radar  system  to  distinguish  between 
two  or  more  targets  on  the  same  bearing  but  at  different  ranges  (Figure  15).  The 
degree  of  range  resolution  depends  on  the  width  of  the  transmitted  pulse,  the 
types  and  sizes  of  targets  and  the  efficiency  of  the  receiver  and  indicator.  Pulse 
width  is  the  primary  factor  in  range  resolution. 
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Figure  1 5.  Range  Resolution.  The  left  pair  of  targets  in  closer  distance  is 

detected,  but  the  right  one  at  a  greater  distance  is  not  (From  Radar 
Tutorial,  www.radartutorial.eu). 


2.  Related  Studies 

In  this  section,  we  provide  an  overview  of  studies  that  investigated  how 
the  type  of  UAVs  used  can  affect  their  tactics.  For  example,  the  payload  a  UAV 
can  carry  determines  the  way  it  will  be  used.  If  a  UAV  has  long  endurance,  it  will 
be  assigned  to  cover  large  areas;  if  this  is  not  the  case,  then  operational  priorities 
will  have  to  be  set.  Furthermore,  the  platforms  from  which  they  are  launched 
along  with  the  extent  to  which  they  are  affected  by  adverse  weather  conditions 
are  crucial  for  operational  or  tactical  commanders  to  take  into  consideration.  In 
addition,  the  number  of  UAVs  needed  to  complete  the  identification  mission  is 
dependent  upon  the  size  of  the  search  area  and  the  sweep-width  of  a  UAV’s 
sensors  (Washburn,  2002). 

Berner  (2004)  studied  the  effective  use  of  multiple  UAVs  for  the  Navy’s 
SSC  Mission.  More  precisely,  he  studied  how  a  BAMS  UAV  and  VTUAVs 
working  together  can  provide  increased  situational  awareness  in  the  maritime 
environment.  The  best  UAV  combination  is  BAMS  plus  two  or  three  VTUAVs. 
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However,  Berner’s  analysis  showed  that  small  numbers  of  VTUAVs  can  perform 
as  well  without  BAMS  as  they  do  with  BAMS.  For  combinations  with  multiple 
UAVs,  BAMS  proves  to  be  a  valuable  asset  that  not  only  reduces  the  number  of 
missed  classifications,  but  greatly  improves  the  amount  of  coverage  on  all 
contacts  in  the  maritime  environment.  BAMS  tactics  have  less  effect  than  the 
mere  presence  of  BAMS  itself. 

Lalis  (2007),  using  agent-based  simulation  (MANA),  identified  that  the 
most  important  factors  when  planning  UAV  operations  are  the  size  of  the  area 
being  covered,  the  Time  on  Task  (ToT)  and  the  detection  range  of  the  UAV.  He 
also  assumed  that  the  UAV’s  probability  of  detection  is  perfect  (Probability  of 
Detection^ )  when  a  target  is  within  a  radius  of  2-18  nm  (“cookie-cutter” 
approach).  This  led  to  the  conclusion  that  the  length  of  the  shoreline  has  no 
impact  on  the  target’s  survivability  because  UAVs  fly  over  the  whole  coastline. 
Lalis  also  studied  the  effect  of  the  UAV’s  three  search  patterns  on  target 
detection  and  concluded  that  “if  ToT  is  not  under  consideration,  then  the  three 
patterns  are  not  different."  Conversely,  other  study  suggests  that  “the  patterns 
that  UAVs  fly  have  a  direct  effect  on  the  coverage  area  and  probability  of 
detection  of  contacts  of  interest”  (Gottfried,  2004). 

McMindes  (2005)  studied  the  impact  of  a  wide  variety  of  factors,  such  as 
UAV  speed,  stealth,  altitude  and  sensor  range,  as  well  as  enemy  force  sensor 
ranges,  probability  of  kill,  array  of  forces  and  numerical  strength  on  the  UAV’s 
survivability.  He  concluded  that  a  speed  between  135  and  225  kn  increases  the 
UAV’s  survivability.  The  exception  to  speed’s  dominance  is  in  the  face  of 
extremely  high  capability  enemy  assets.  In  this  case,  stealth  becomes  more 
important  than  speed  alone.  However,  the  interactions  indicate  that  as  both 
speed  and  stealth  increase,  speed  yields  a  faster  return  on  overall  survivability, 
and  that  speed  mitigates  increased  enemy  capabilities.  Concerning  altitude, 
increased  altitude  produces  higher  mean  survivability  as  well  as  decreased 
variability. 
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Yildiz  (2009)  studied  the  use  of  mini-UAVs  in  border  security  and  found 
them  to  be  beneficial  in  capturing  the  illegal  entrants  and  thus  could  potentially 
provide  more  secure  borders.  Adequate  manpower  and  a  reliable  communication 
scheme  to  compose  a  COP  emerged  as  the  most  important  factors. 

G.  TASK  ANALYSIS 

EO/IR  systems  produce  images  that  must  be  scanned  for  recognizable 
patterns  of  targets.  Computers  have  an  ability  to  scan  imagery  continuously  for 
long  periods  of  time  without  risk  of  degraded  performance  due  to  boredom  or 
fatigue.  Humans,  on  the  other  hand,  are  characterized  by  a  greater  capability  to 
pattern  match  visual  images,  but  they  are  susceptible  to  performance  errors. 
Combining  computer  and  human  analysis  allows  for  the  benefits  of  each 
resource  to  be  utilized  resulting  in  enhanced  capabilities.  The  analysis  of  EO/IR 
images  can  be  done  at  the  location  of  the  sensor  or  at  a  central  location  (Brown 
&  Schulz,  2009). 

In  this  section  we  conduct  a  task  analysis,  because  it  is  the  process  that 
describes  the  user’s  task  in  detail  and  helps  stakeholders  to  design  a  system  that 
supports  users  in  doing  this  task  in  the  most  effective  way.  Task  analysis  has 
long  been  used  as  a  fundamental  step  in  system  design  and  is  useful  for 
examining  new  or  existing  systems.  Understanding  the  process  and  task 
structure  of  target  detection  may  shed  more  light  on  how  surveillance  can  be 
conducted. 

Because  the  surveillance  task  used  in  this  study  is  very  complex  and 
incorporates  inputs  from  and  activities  performed  by  many  field  users,  we 
subdivide  it  into  four  distinct  subtasks.  These  subtasks  are:  communications, 
search,  detection  and  identification.  The  subdivision  of  a  system  into  autonomous 
tasks  is,  in  general,  a  powerful  technique  for  studying  a  complex  system.  In  the 
case  of  distributed  surveillance  systems,  the  subdivision  of  a  system  into 
elementary  modules  is  necessary  for  the  distribution  of  intelligence  (Marcerano 
et.  al,  2001). 
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Note  that  the  following  task  analysis  (Table  8)  refers  to  a  high  level  of 
command  because  too  many  details  would  be  out  of  the  scope  of  this  study. 

Table  8.  Maritime  surveillance  task  analysis. 


No 

Task 

Subtask/  user  action 

System  Actions 

1 

Establish  communications 

1.1  Operate  the  appropriate 
communication 
devices 

Telephones,  VHF,  UHF, 
Link  11-14 

1 .2  Get  familiar  with  the 
functionalities 
and  the  military 
procedures 

1 .3  Take  measures  against 
information 
leakage 

Implement  corresponding 
doctrines 

1.4  Set  up  communication 
with  other 
partners 

Set  up  communication 
with  your 
collaborators 
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Detection 


3.1  Record  time  and  position  of 
_ the  detected  target 

3.2  Match  the  detection  with 

information  for 
potential  target  in 
the  area 

3.3  Search  more  thoroughly  in 

the  area  close  to  it 
for  additional 
targets 

3.4  Verify  that  it  is  not  a 

pseudo-target 

3.5  Report  its  time  and  speed 


Are  you  expecting  a  target 
in  this  area  or 
not? 

Approach  the  target  to  have 
better  visual 
contact  and 
increase  radar 
resolution 

Tune  your  device  for 
optimum 
performance 


4.1  Correlate  information 

provided  by  other 
agencies  to  identify 
the  detected  target 


4.2  Identify  visually  the 

detected  target 


4.3  Identify  the  target  using 

electronic  devices 


4.4  Identify  the  target  by  its 
electronic 
emissions  (if 
available) 


4.5  Report  its  identity 


4.6  Keep  continuous  contact 
with  the  identified 
target 


4.7  Record  its  electronic 

emissions  in  the 
data  base  (if 
available) 


What  type  of  target  am  I 

expecting  in  this 
area? 


1 


Identify  positively  the  target 
using  devices 
(electro  optical  , 
l/R) 


IFF,  monitor  AIS  system, 
ESM 


Enrich  the  corresponding 
data  base  for 
future  use 
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III.  MODEL  DEVELOPMENT 


A.  MODEL  SELECTION 

Models  are  used  to  represent  and  describe  the  behavior  of  the  system 
being  modeled.  Models  assist  users  in  gaining  a  better  understanding  of  the  real 
world,  and  in  most  cases  they  present  real  systems  in  a  simplified  way.  In 
general,  models  are  the  most  cost-effective  way  to  study  a  system.  They  are  also 
a  risk-free  solution  in  cases  where  dealing  with  the  actual  system  itself  might  put 
the  user  in  danger.  Another  benefit  of  models  is  their  ability  to  reduce  the  time, 
space,  and  means  needed  to  study  a  system  (Sanchez,  2007). 

“A  system  is  defined  to  be  a  collection  of  entities  that  act  and  interact 
together  toward  the  accomplishment  of  some  logical  end.  In  practice,  what  is 
meant  by  system  depends  on  the  objectives  of  a  particular  study”  (Law,  2007). 
Systems  can  be  classified  in  a  variety  of  ways.  In  this  study  we  focus  on 
simulation  solutions  to  study  the  efficacy  of  using  UAVs  to  monitor  a  complex 
marine  environment. 


Figure  16.  Ways  to  study  a  system  (From  Law,  2007). 
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Analytical  solutions  are  preferred  when  the  elements  that  compose  the 
model  are  relatively  simple.  Using  such  an  approach,  it  is  possible  to  obtain  a 
closed-form  solution  and  get  a  precise  answer.  These  methods  may  involve 
algebra,  calculus,  probability  theory,  linear  programming,  decision  analysis, 
Markov  chain  analysis  and  queuing  theory  (Law,  2007,  pp.  3-6;  Berner,  2004). 
However,  when  a  system  is  highly  complex  and  an  analytical  solution  is 
infeasible,  we  use  simulation  solutions,  instead.  In  a  simulation,  we  use  a 
computer  to  evaluate  a  model  numerically,  and  data  are  gathered  to  estimate  the 
desired  true  characteristics  of  the  model  (Law,  2007). 

Simulation  has  both  advantages  and  disadvantages,  and  every 
user/analyst  should  be  aware  of  them.  By  simulating  a  system,  we  numerically 
exercise  the  model  to  investigate  how  the  inputs  affect  the  output  measures  of 
performance.  That  means  that  we  are  capable  of  making  predictions  of  input 
changes.  Additionally,  visualization  can  give  a  good  understating  of  a  system  and 
allow  us  to  study  its  behavior  in  detail.  On  the  other  hand,  a  simulation,  as  are  all 
models,  is  limited  by  its  assumptions.  Some  assumptions  may  be  reasonable,  but 
some  may  be  implemented  solely  to  make  the  model  less  complex.  In  any  case, 
these  assumptions  are  set  by  the  user,  and  in  altering  these  assumptions,  we 
change  the  output,  yielding  a  less  robust  solution.  Furthermore,  in  some  cases,  it 
can  be  time  consuming  to  try  to  build  complex  models  with  debatable  results. 

B.  MAP-AWARE  NONUNIFORM  AUTOMATA 

In  this  study,  the  Map-Aware  Non-uniform  Automata  (MANA)  Version  V 
software  is  used  as  the  simulation  tool.  This  section  explains  the  reasons  that 
this  model  was  chosen  and  describes  briefly  its  major  features. 

1.  Overview  of  MANA 

MANA  is  an  agent-based,  time-stepped  simulation  modeling  environment. 

Agent-based  models  seem  to  be  the  most  appropriate  for  the  current  study 

because  of  their  ability  to  describe  the  behavior  of  individual  entities  in  complex 

systems.  MANA  is  a  Discrete  Time  Simulation  (DTS),  which  uses  a  fixed  time 
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increment  as  the  time  advance  method  (Alrowaei,  2011).  DTS  is  the  most 
commonly  used  time  advance  mechanism  in  combat  simulation  and  agent-based 
models  (Macal,  2010,  pp.  371-382).  The  fact  that  MANA  also  has  been  used  in 
numerous  Naval  Postgraduate  School  theses  and  in  other  studies  in  the  military 
domain  indicates  its  usefulness. 

MANA  has  been  built  upon  two  key  ideas.  The  first  idea  describes  how  the 
model  will  contain  entities  that  are  controlled  by  decision-making  algorithms. 
These  entities  (agents)  interact  with  each  other,  as  well  as  the  environment  in 
which  they  operate,  and  make  their  own  decisions  based  upon  the  “personality” 
the  modeler  gave  to  them.  The  second  idea  has  to  do  with  the  simplicity  of  the 
model.  The  creators  of  MANA  claim  that  more  detailed  models  are  not 
necessarily  better  (McIntosh,  2007).  Additionally,  the  non-linear  nature  of 
equations  describing  real  situations  is  sensitive  to  initial  conditions  resulting  in 
non-robust  solutions  (McIntosh,  2007).  Perhaps,  this  is  the  biggest  benefit  of 
MANA;  although  it  is  designed  to  describe  complex  systems,  like  any  other 
simulation  tool,  it  does  so  with  comparatively  simple  models. 

MANA  is  intended  to  address  a  wide  spectrum  of  problems  and  was 
designed  for  use  in  scenario  exploring.  That  is,  the  modeler  can  vary  the 
parameters  of  the  model,  even  while  it  is  running,  and  observe  the  output.  This 
feature  is  a  great  benefit  for  the  analysts  because  they  have  the  capability  to 
explore  the  relative  importance  of  a  variable  on  the  final  result.  Furthermore, 
MANA  is  user-friendly  and  allows  the  modeler  to  create  his/her  own  scenario  in  a 
short  time  using  a  very  simple  and  understandable  interface.  Building  the 
appropriate  scenario,  the  user  can  explore  how  the  agents  perform  and  gain 
insights  about  the  potential  outcomes. 

2.  MANA  Characteristics 

In  this  section,  the  innovative  features  of  MANA  compared  to  other 
models,  along  with  its  basic  characteristics,  is  briefly  discussed.  The  most 
important  elements  of  MANA  and  its  concepts  are  presented  to  provide  the 
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reader  with  information  most  relevant  to  this  study.  For  further  details  see 
McIntosh  (2007). 

•  Agent  is  the  individual  entity  that  “operates”  in  the  scenario.  Each 
entity  has  certain  personality  traits  that  drive  it  toward  or  away  from 
other  entities,  waypoints  or  areas  with  specific  characteristics  (i.e. , 
areas  that  provide  cover  or  areas  in  which  it  is  easier  to  move).  An 
agent  is  also  equipped  with  sensors,  weapons  and  speed  of 
movement,  all  of  them  with  capabilities  arranged  by  the  modeler.  It 
may  also  have  established  communication  links  with  other  entities. 

•  Squad  is  a  group  of  agents  of  any  size,  as  defined  by  the  user. 
Agents  in  a  squad  share  the  same  propensities,  the  same 
capabilities  and  the  same  SA  Map.  In  a  naval  scenario,  like  those 
studied  in  this  thesis,  a  squad  may  be  considered  as  a  squadron  of 
ships  of  the  same  type. 

•  Battlefield  is  the  area  in  which  the  scenario  is  taking  place.  Unlike 
previous  versions,  in  MANA  V  battlefield  distances  and  agent 
speeds  can  be  specified  directly  in  terms  of  real-world  distances 
and  times.  The  battlefield  time  interval  can  be  set  by  the  user.  The 
shorter  the  time  interval,  the  more  accurate  models  we  have,  but 
more  calculations  are  needed  resulting  in  more  time  consuming 
simulation  runs.  A  background  map  can  be  loaded  for  illustrative 
purposes  without  any  impact  on  the  simulation  run.  In  contrast, 
terrain  maps  can  be  used  to  identify  terrain  features.  These  maps 
(Figure  17)  are  standard  Windows  bitmap  files,  which  use  colors  to 
impact  the  agent  movement  in  the  battlefield.  Apart  from  the  default 
colors,  the  user  can  create  various  colors  to  capture  terrains  with 
different  characteristics  with  respect  to  three  factors:  Going,  Cover 
and  Concealment  (Table  9).  These  factors  act  as  multipliers  to 
movement  speed,  hit  rate  and  sensor  detection,  respectively.  For 
example,  if  an  area  has  been  defined  to  have  Going=0,  Cover=1 
and  Concealments  then  the  agent  who  will  be  there  will  not  be 
able  to  move,  he  cannot  be  shot  at,  and  no  one  can  see  him. 
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Figure  17.  Background  Map  (left)  used  in  a  scenario  in  MANA.  Terrain  Map  (right) 
for  the  same  scenario.  The  values  that  have  been  set  for  the  colors  in 
the  Terrain  Map  are  noted  in  Table  9. 


Table  9.  Values  for  Going/Cover/Concealment  of  the  Terrain  Map  depicted 

in  Figure  17. 


Name 

Going 

Cover 

Concealment 

Color 

Sea 

1.00 

0.00 

0.00 

Black 

Land 

0.10 

0.00 

0.00 

Grey 

Near  Land 

0.75 

0.50 

0.50 

Light  Green 

•  Trigger  State  feature  provides  MANA  with  additional  flexibility.  As 
the  scenario  evolves,  certain  events  may  occur  to  an  agent  (or  its 
squad).  These  events  might  be  the  presence  of  an  enemy  in  its 
vicinity,  the  arrival  at  a  waypoint,  a  shot  taken,  an  agent  injury  and 
many  more.  If  an  event  happens,  the  modeler  can  define  a  new  set 
of  agent  personality  weightings,  speed,  sensor  capabilities  or  any 
other  parameter  that  “describes”  this  agent.  The  modeler  is  able 
also  to  define  the  time  that  the  agent  shifts  in  the  new  “trigger 
state,”  as  well  as  a  potential  sequence  of  trigger  states,  enhancing 
the  agent’s  behavior  which  might  emerge  in  a  scenario. 

•  Sensors  are  critical  features  for  the  agents.  Each  entity  can  carry 
up  to  six  sensors,  either  simple  or  advanced.  “Simple  sensors  only 
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provide  a  cookie-cutter  model  of  detection  and  classification  while 
advanced  sensors  allow  the  specification  of  a  range-integration 
time  profile  for  both”  (McIntosh,  2007).  The  term  “cookie-cutter” 
means  every  visible  contact  within  the  detection  range  will  be 
detected  (or  classified  for  the  case  of  a  classification  range).  On  the 
other  hand,  the  advanced  sensor  type  uses  times  between 
detections  with  respect  to  the  range  solely  for  the  detection  task. 
For  the  classification,  it  uses  range-dependent  probability  of 
classifying  detecting  events.  It  is  important  to  note  that  detection  is 
a  prerequisite  for  classification. 

•  Situational  Awareness  (SA)  Map  is  one  of  the  innovative  elements 
of  MANA.  It  is  graphical  representation  of  a  squad’s  perception  of 
the  battlefield.  That  is,  it  is  a  map  that  depicts  the  contacts 
(classified  or  not)  of  which  the  squad  is  aware.  It  represents  intra¬ 
squad  communications  whereby  squad  members  actively  share 
information,  as  it  would  be  expected  in  a  real  combat  team.  The 
modeler  can  also  define  the  time  taken  for  information  to  get  from 
the  agents’  sensors  to  SA  Map.  This  definition  models  potential 
time  delays  in  information  sharing. 

•  Inorganic  SA  Map  represents  information  obtained  remotely  from 
other  squads,  as  opposed  to  the  squad’s  own  sensors.  The 
modeler  defines  which  squads  could  communicate  as  well  as  the 
quality  of  the  communication  with  respect  to  several  different 
parameters  such  as  Latency,  Reliability,  Capacity  and  Filtering  of 
information.  Figure  18  is  a  snapshot  of  an  SA  Map  and  an  Inorganic 
SA  Map  for  the  same  time  of  a  simulation  run.  The  main  difference 
between  the  two  is  that,  in  the  second,  only  contacts  coming  in  from 
remote  squads  will  be  displayed.  In  reality,  the  agent  is  aware  of 
both  the  pictures.  With  all  these  features,  MANA  is  an  appropriate 
tool  to  study  aspects  of  network  centric  warfare. 
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Figure  18.  Snapshot  of  SA  Map  (left)  and  Inorganic  SA  Map  (right),  concerning 
Squad  1 ,  for  the  same  time  of  the  simulation  run.  Fellow  squad 
members  are  represented  by  triangles,  other  friendly  targets  are 
represented  by  inverted  triangles,  enemies  are  represented  by  (red) 
squares  and  neutral  targets  are  represented  by  (blue)  inverted 
squares.  Unknown  contacts  are  represented  by  white  rectangles. 


•  Fuel  is  one  more  innovative  property  of  MANA.  It  does  not 
necessarily  refer  to  fuel,  food  or  other  supplies,  but  it  can  be  used 
to  record  interactions  with  other  agents  or  to  assist  in  shifting  into 
certain  trigger  states.  The  modeler  can  define  several  parameters 
such  as  refuel  range,  probability  to  refuel  enemy/  friend/  neutral  and 
refuel  rate.  This  feature  is  used  extensively  in  our  model  and  will  be 
explained  in  detail  later. 
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C.  MODEL  IMPLEMENTATION 
1.  Models  in  MANA 

As  described  in  previous  chapters,  the  Aegean  Sea  is  an  extremely 
complex  environment  because  it  consists  of  areas  either  of  “open  sea,”  where  a 
ship  can  sail  for  about  50  nautical  miles  without  the  “coverage”  of  nearby  islands, 
or  areas  where  “many  islands”  are  very  close  to  one  another.  These  rare 
geographical  conditions  compose  a  challenging  environment  operationally.  The 
effectiveness  of  UAVs  is  studied  separately  in  those  two  environments  creating 
two  different  models,  as  described  in  the  following  sections. 

a.  “Open  Sea”  Model 

The  “Open  Sea”  model  represents  a  typical  area  of  the  Northern 
and  Southern  Aegean  Sea.  In  this  model  (Figure  19),  the  Red  Force  consists  of 
four  surface  vessels.  These  vessels  attempt  to  transport  illegal  immigrants  to  the 
small  island  on  the  Southwest.  To  succeed  in  their  mission  they  have  to  sail  the 
open  sea  without  being  detected  by  the  Blue  Surveillance  System.  To  increase 
the  possibility  of  going  undetected,  they  keep  close  to  nearby  merchant  vessels 
that  have  approximately  the  same  course  with  them.  They  have  no 
communication  capabilities,  but  it  is  assumed  that  they  have  the  same  tactic. 
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Figure  19.  “Open  Sea”  model  implemented  in  MANA. 

The  Blue  Force,  which  has  the  responsibility  to  monitor  the  AOI, 
consists  of  two  ground  radars,  located  on  the  Northwest  and  Southeast  islands 
respectively,  as  well  as  two  Blue  surface  ships  which  patrol  in  two  different 
sectors.  In  variations  of  this  model,  we  study  the  effect  of  one  or  two  UAVs  on  the 
performance  of  the  surveillance  system. 

In  the  area,  there  are  a  large  number  of  merchant  vessels  (yellow 
ships)  that  sail  along  specific  transit  lanes  either  northbound,  or  southbound.  In 
the  Northeast  corner  are  the  Dardanelles  Straights,  which  link  the  East 
Mediterranean  with  the  Black  Sea,  and  are  usually  characterized  by  high  traffic. 
Additionally,  there  are  randomly  scattered  fishing  vessels  (green  ships), 
something  very  common  in  the  area. 

b.  “Many  Islands”  Model 

The  “Many  Islands”  model  represents  a  typical  area  of  the  Central 
and  Southeast  Aegean  Sea.  In  this  model  (Figure  20),  the  Red  Force  consists  of 
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four  surface  vessels  which,  again,  attempt  to  transport  illegal  immigrants  beyond 
the  islands  on  the  West.  To  succeed  in  their  mission  they  have  to  sail  westbound 
without  being  detected  by  the  Blue  Surveillance  System.  To  increase  the 
possibility  of  going  undetected,  they  do  not  sail  directly  to  the  final  destination; 
instead  they  sail  close  to  the  shoreline  during  their  route.  They  have  no 
communication  between  them,  but  it  is  assumed  that  they  have  the  same  tactic. 
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Figure  20.  “Many  Islands”  model  implemented  in  MANA. 


The  Blue  Force,  which  is  responsible  for  monitoring  the  AOI, 
consists  of  two  ground  radars  located  on  the  West  and  Southeast  islands, 
respectively,  as  well  as  two  Blue  surface  ships  that  patrol  in  two  different  sectors. 
In  variations  of  this  model,  we  study  the  effect  of  one  or  two  UAVs  on  the 
performance  of  the  surveillance  system. 

In  the  area,  there  are  a  large  number  of  merchant  vessels  (yellow 
ships)  that  sail  in  various  routes.  This  represents  the  movement  of  merchant 
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vessels,  cruise  ships  or  coastal  shipping.  Additionally,  there  are  randomly 
scattered  fishing  vessels  (green  ships),  something  very  common  in  the  area. 

2.  Assumptions 

Every  model  is  subject  to  assumptions  which  limit  it  in  specific 
circumstances.  The  models  that  are  demonstrated  in  this  study  have  the 
following  assumptions: 

•  No  intelligence  from  other  resources  is  taken  into  account.  The 
Blue  and  Red  forces  are  based  only  on  the  information  taken  from 
their  own  sensors. 

•  The  Red  ships  cannot  communicate  with  each  other  to  exchange 
information  about  the  adversaries. 

•  The  size  of  the  Red  surface  vessels  is  expected  to  be  medium, 
such  as  fast  patrol  boats  or  fast  motor  boats.  This  factor  affects  the 
detection  range,  as  explained  in  Chapter  II. 

•  The  Red  ships  do  not  start  sailing  towards  their  goal  unless  they 
meet  a  merchant  vessel  that  is  sailing  to  the  same  area  (“Open 
Sea”  model). 

•  The  Blue  UAVs  and  Blue  ships  get  close  to  the  enemy  and  remain 
there  for  a  period  of  time  only  once.  Operationally,  it  is  reasonable 
for  Blue  sensors  to  gather  information  about  a  specific  target  one 
time.  When  a  target  has  been  lost,  but  we  expect  it  to  be  in  a 
specific  area,  we  are  satisfied  if  only  one  of  our  sensors  locates  it. 

•  Blue  ships  and  UAVs  spend  time  close  to  enemy  targets  only  to 
collect  information  (photos,  electronic  emissions,  etc.).  This  is  not 
necessary  for  merchant  and  fishing  vessels  because  once  it  is 
known  that  the  contact  is  neutral,  the  Blue  units  immediately  move 
to  the  next  unidentified  contact  or,  if  there  is  none,  along  their  patrol 
route. 

3.  Measures  of  Effectiveness  (MOE) 

It  is  desirable  for  the  Blue  Force  to  monitor  the  AOI  consistently.  That 
means  that  we  want  them  to  detect  and  identify  as  many  targets  as  they  can  but 
also  to  keep  contact  with  the  targets  as  much  time  as  possible.  Detecting  and 
then  losing  a  target  gives  only  a  piece  of  information  on  the  target’s  existence 
and  nothing  about  its  current  location.  For  this  reason,  it  is  considered  that  a 
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more  robust  MOE  to  evaluate  the  system’s  performance  is  the  “Proportion  of  time 
Red  Ships  are  positively  identified”  and  “Proportion  of  time  Merchant  Vessels  are 
positively  identified.”  We  expect  the  two  MOEs  to  be  highly  correlated,  but  we 
also  want  to  investigate  whether  the  surveillance  system  performs  differently  with 
respect  to  the  type  of  contact. 

4.  Agents  Description 

As  mentioned  earlier,  a  squad  is  a  group  of  agents  that  have  the  same 
characteristics.  Every  agent  has  a  unique  allegiance  that  depicts  the  agent’s  side 
during  the  model  run.  It  can  be  on  the  Blue  side  (Allegiance  1),  on  the  Red  side 
(Allegiance  2)  or  neutral  (Allegiance  0).  MANA  offers  additional  features,  such 
“Threat  Level”  and  “Class,”  so  that  differentiations  among  the  agents  can  be 
expressed.  All  the  agents  used  in  the  “Open  Sea”  model,  along  with  their 
allegiance,  threat  level  and  class,  are  shown  in  Table  10.  The  same  agents  with 
similar  characteristics  are  also  used  in  the  “Many  Islands”  model. 

Table  10.  Agents  used  in  the  “Open  Sea”  and  “Many  Islands”  models 


Agent’s  Name 

Agent’s 

Agent’s  Threat 

Agent’s 

Allegiance 

Level 

Class 

Blue  UAV 

1 

3 

7 

Blue  Ship 

1 

3 

1 

Blue  Ground  Radar  1 

1 

3 

3 

Blue  Ground  Radar  2 

1 

3 

4 

Blue  Operational  Centre 

1 

2 

0 

Red  Ship 

2 

3 

0 

Merchant  Vessel 

0 

3 

0 

Fishing  Vessel 

0 

1 

0 
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a.  Blue  Ships 

In  every  model  there  are  two  surface  Blue  ships.  At  the  beginning 
of  each  scenario,  the  Blue  ships  patrol  along  a  specific  route  covering  different 
sectors.  Together,  along  with  the  other  Blue  sensors  described  in  this  section, 
they  cover  all  the  AOI.  The  patrol  route  is  focused  near  the  coastline,  as  well  as 
inside  the  territorial  waters,  to  detect  potential  hidden  targets.  They  exchange 
information  with  the  Operational  Centre  to  have  improved  SA.  Initially,  a  Blue 
ship  has  a  higher  propensity  to  move  towards  unknown  contacts  in  order  to 
identify  them  (Figure  21).  If  it  meets  an  enemy  (Red  ship),  it  sails  towards  it 
(“Enemy  Contact  3”  state)  and  stays  close  to  it  for  more  than  15  minutes  (1000 
time  steps). 


Figure  21 .  Personalities  tab  of  the  Blue  ship  agent  for  the  default  state. 


This  time  is  provided  to  collect  all  the  information  needed  (photos, 
electronic  emissions,  etc.).  Nothing  is  capable  of  interrupting  this  mission.  After 
this  time  has  passed,  the  Blue  ship  shifts  to  the  “Spare  1”  state  where  it  shoots  at 
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the  enemy  (with  zero  probability  of  success).  This  is  only  to  help  the  Red  ship  to 
shift  to  the  “Shot  At”  state,  so  that  the  Blue  unit  breaks  lock  of  it.  Once  the  Blue 
ship  stops  monitoring  the  enemy  target,  it  sails  towards  other  unidentified 
contacts  or,  if  there  is  none,  along  its  patrol  route.  To  avoid  engaging  on  the 
same  target  with  other  friendly  forces,  the  Squad  Friendly  Awareness  has  been 
set  to  -100.  Finally,  the  Blue  ship  is  equipped  with  a  detecting  sensor.  In  this 
study  the  Advanced  Sensor  Model  is  used.  The  detection  range  has  been  divided 
into  five  zones,  and  each  one  has  different  detection  probability.  It  is  more 
accurate  to  consider  that  a  target  closer  to  the  detecting  sensor  has  a  higher 
probability  of  being  detected  than  a  target  close  to  the  maximum  range.  The 
same  method  is  applied  for  the  Classification  Probabilities  (Figure  22).  Finally, 
when  a  Blue  ship  identifies  a  target,  it  “refuels”  it  with  a  rate  of  1  fuel  unit  per  time 
step.  At  the  end  of  each  simulation  run,  the  remaining  “fuel”  is  used  to  calculate 
the  time  a  target  was  monitored. 


Figure  22.  Sensors  tab  of  the  Blue  ship  agent. 
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b. 


Blue  UAV 


Every  model  is  studied  having  one  or  two  UAVs  available.  The  Blue 
UAVs  have  similar  behavior  to  the  Blue  ships.  If  one  UAV  is  available,  it  starts  its 
patrol  from  the  Northwest  area  (Figure  23),  where  the  first  Blue  ship  is  located. 
The  patrol  route  is  focused  near  the  coastline,  as  well  as  inside  territorial  and 
international  waters,  to  detect  potential  hidden  targets.  If  a  second  UAV  is 
available,  it  starts  from  the  Southeast  area  in  the  “Open  Sea”  model,  where  the 
second  Blue  ship  is  located  (in  the  case  of  the  “Many  Islands”  model,  it  starts 
from  the  Southwest  area).  In  that  case,  the  two  UAVs  cover  different  sectors. 
Together,  along  with  the  other  Blue  sensors  described  in  this  section,  they  cover 
all  the  AOI.  They  pass  information  to  the  Operational  Centre  and  receive 
information  from  the  nearest  ground  radar. 
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Figure  23.  Map  Tab  of  the  Blue  UAV  agent  in  the  “Many  Island”  model  with  one 

UAV  available. 
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Initially,  the  Blue  UAV  has  a  higher  propensity  to  move  towards 
unknown  contacts  in  order  to  identify  them.  If  it  meets  an  enemy  (Red  ship),  it 
sails  towards  it  (“Enemy  Contact  3”  state)  and  stays  close  to  it  for  more  than  15 
minutes  (1000  time  steps).  This  time  is  provided  to  collect  all  the  information 
needed  (photos,  electronic  emissions,  etc.).  Nothing  is  capable  of  interrupting 
this  mission.  After  this  time  is  passed,  the  Blue  UAV  shifts  to  the  “Spare  1”  state, 
where  it  shoots  at  the  enemy  (with  zero  probability  of  success).  This  is  only  to 
help  the  Red  ship  to  shift  to  the  “Shot  At”  state,  so  that  the  Blue  unit  breaks  lock 
of  it.  Once  the  Blue  UAV  stops  monitoring  the  enemy  target,  it  sails  towards  other 
unidentified  contacts  or,  if  there  is  none,  along  its  patrol  route.  To  avoid  engaging 
on  the  same  target  with  other  friendly  forces,  the  Squad  Friendly  Awareness  has 
been  set  to  -100.  Finally,  the  Blue  UAV  is  equipped  with  a  detecting  sensor.  In 
this  study  the  Advanced  Sensor  Model  is  used.  The  detection  range  has  been 
divided  into  five  zones,  and  each  one  has  different  detection  probability.  The 
same  method  is  applied  for  the  Classification  Probabilities.  Finally,  when  a  Blue 
UAV  identifies  a  target,  it  “refuels”  it  with  a  rate  of  one  fuel  unit  per  time  step.  At 
the  end  of  each  simulation  run,  the  remaining  “fuel”  is  used  to  calculate  the  time 
a  target  was  monitored. 

c.  Blue  Ground  Radars 

The  Blue  ground  radars  have  a  limited  FOV  of  270  degrees.  This  is 
reasonable  because  none  of  the  current  ground  radars  in  the  Aegean  Sea  has  a 
peripheral  “visible”  sector.  They  send  all  the  information  about  the  targets 
detected  to  the  Blue  Operational  Centre.  They  also  provide  information  to  the 
available  UAVs.  The  Advanced  Mode  for  the  sensors  is  used.  The  maximum 
detection  and  classification  ranges  are  subdivided  into  five  zones,  and  each  one 
has  different  probability  of  detection/classification.  This  captures  the  intuitive 
opinion  that  if  a  contact  is  closer  to  the  radar,  it  has  more  chances  to  be 
detected.  While  the  radar  has  a  contact  under  positive  identification,  it  “refuels”  it, 
and  this  helps  us  to  keep  track  of  the  time  the  contact  is  classified. 
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d. 


Blue  Operational  Centre 


The  role  of  the  Blue  Operational  Centre  is  to  gather  the  information 
from  the  Blue  sensors  and  send  them  to  the  Blue  ship  (Squad  1)  and  Blue  UAV 
(Squad  7)  (Figure  24).  The  Operational  Centre  does  not  have  detection 
capabilities  itself.  In  reality,  it  can  be  located  either  away  from  the  AOI  or  onboard 
a  Command  ship. 


Figure  24.  Inter  squad  SA  table  of  the  Blue  Operational  Centre. 

e.  Red  Ships 

In  every  model  there  are  four  Red  ships.  Depending  on  the  model, 
they  have  different  tactics.  In  the  “Open  Sea”  model,  they  approach  the  closest 
merchant  vessel  and  stay  close  to  it  for  more  than  2.5  hours  (10,000  time  steps) 
while  transiting  the  open  sea.  During  this  tactic,  they  increase  their  concealment 
in  order  to  avoid  detection  by  the  Blue  surveillance  system.  In  the  “Many  Islands” 
model,  the  Red  ships  prefer  to  stay  close  to  the  shoreline  in  order  to  reach  their 

final  destination.  Again,  using  this  tactic,  they  increase  their  concealment. 
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When  the  Red  ships  have  been  identified,  they  shift  to  the  “Shot  At” 
state  and  become  invisible  allowing  the  Blue  UAV/ships  to  move  to  the  next 
unidentified  contact.  They  stay  in  this  state  for  30  minutes  (1800  time  steps), 
where  they  are  considered  as  identified.  Operationally,  this  is  reasonable 
because  in  such  a  short  time  it  is  extremely  difficult  to  mislead  the  surveillance 
system  and  avoid  the  detection.  While  the  vessels  are  under  positive 
identification,  they  consume  “fuel,”  and  this  helps  us  to  keep  track  of  the  time  the 
contact  is  classified.  They  do  not  have  communication  and  weapon  capabilities. 
They  are  equipped  with  a  detection  sensor  operating  in  Simple  Mode  (cookie- 
cutter),  as  depicted  in  Figure  25. 


Figure  25.  Sensors  tab  of  the  Red  ship  agent. 


f.  Merchant  Vessels 

Merchant  vessels  are  randomly  distributed  in  the  AOI.  Their  routes 
are  consistent  with  the  real  merchant  shipping  lanes  in  the  Aegean  Sea.  They  do 
not  have  communication  and  weapon  capabilities.  Once  they  are  classified  by 
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the  Blue  sensors,  they  are  assumed  as  such  for  the  next  hour.  For  this  time 
period,  they  shift  to  the  “Refuel  by  Friend”  state  and  increase  their  concealment 
to  100  percent.  That  makes  them  invisible  allowing  the  Blue  Ship/UAV  to  move  to 
the  next  unidentified  contact.  When  the  one-hour  period  expires,  they  have  to  be 
revisited  in  order  to  be  re-identified.  Operationally  this  is  reasonable  because 
merchant  vessels  usually  keep  constant  speed  and  heading,  and  this  fact 
facilitates  the  users  of  the  surveillance  sensors  to  keep  track  of  them.  While  the 
vessels  are  under  positive  identification,  they  consume  “fuel,”  and  this  helps  us  to 
keep  track  of  the  time  the  contact  is  classified. 

g.  Fishing  Vessels 

Fishing  vessels  are  randomly  distributed  in  the  AOI.  They  remain 
constant  throughout  the  scenarios.  They  do  not  have  communication  and 
weapon  capabilities.  Once  they  are  classified  by  the  Blue  sensors,  they  are 
assumed  as  such  for  the  next  two  hours.  For  this  time  period,  they  shift  to  the 
“Refuel  by  Friend”  state  and  increase  their  concealment  to  100  percent.  That 
makes  them  invisible  allowing  the  Blue  Ship/UAV  move  to  the  next  unidentified 
contact.  When  the  two-hour  period  expires,  they  have  to  be  revisited  in  order  to 
be  re-identified.  Operationally  this  is  reasonable  because  fishing  vessels  remain 
approximately  at  the  same  position  for  long  periods  of  time  without  trying  to 
mislead  the  surveillance  system.  While  the  vessels  are  under  positive 
identification,  they  consume  “fuel,”  and  this  helps  us  to  keep  track  of  the  time  the 
contact  is  classified. 


65 


THIS  PAGE  INTENTIONALLY  LEFT  BLANK 


66 


IV.  DESIGN  OF  EXPERIMENT 


A.  EXPERIMENTAL  SETUP  AND  FACTORS 

This  section  provides  a  brief  description  of  the  experimental  setup  that  is 
followed  in  this  study.  Additionally,  the  factors  that  are  varied  in  the  Design  of 
Experiment  (DOE),  along  with  their  ranges,  are  presented. 

1.  Experimental  Setup 

In  this  study,  two  different  scenarios  are  studied.  The  first  is  the  “Open 
Sea”  scenario  in  which  the  enemies  have  to  transit  long  distances  without  the 
coverage  of  nearby  islands  to  reach  their  final  destination.  This  scenario 
represents  the  situation  in  the  Northern  and  Southern  Aegean  Sea.  The  second 
scenario,  which  is  found  in  the  Central  and  Southeast  Aegean  Sea,  is  called 
“Many  Islands”  because  it  is  characterized  by  a  large  number  of  islands  being 
close  to  each  other.  The  Red  ships  try  to  take  advantage  of  this  situation.  The 
two  scenarios  are  studied  separately,  and  the  conclusions  of  both  will  shed  light 
on  a  part  or  the  whole  AOI. 

The  two  scenarios  explore  different  combinations  of  Blue  sensors  and  Red 
tactics.  Concerning  the  Blue  forces,  the  current  surveillance  system  represented 
by  two  ground  radars  and  two  surface  ships  will  be  contrasted  with  some 
proposed  solutions.  The  first  solution  involves  one  UAV  in  addition  to  the  current 
system.  In  the  second  solution,  two  UAVs  support  the  operations  of  the  current 
surveillance  system.  The  last  solution  is  more  challenging  since  the  use  of  two 
UAVs  along  with  two  ground  radars  is  being  studied  without  the  support  of 
surface  ships. 

From  the  enemy’s  perspective,  two  tactics  are  studied.  When  the  Red 

ships  aim  to  reach  a  destination  without  having  the  option  to  sail  close  to  nearby 

islands,  they  approach  transiting  merchant  vessels  and  keep  close  to  them,  as 

much  as  they  can.  This  tactic  is  captured  into  the  “Open  Sea”  scenario.  In 

contrast,  when  the  environment  is  characterized  by  many  islands  in  close 
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distances,  the  Red  ships  try  to  take  advantage  of  the  islands.  Sailing  near  to  the 
shoreline,  they  try  to  avoid  being  detected  by  the  adversary’s  sensors.  This 
situation  is  captured  in  the  “Many  Islands”  scenario.  A  combination,  or  further 
investigation,  of  the  enemy’s  tactics  is  not  part  of  this  thesis.  The  experimental 
setup  just  described  is  shown  in  Table  1 1 . 

Table  1 1 .  Experiment  setup. 


“Open  Sea”  Scenario 

“Many  Islands”  Scenario 

Blue  Sensors 

Combinations 

no  UAV 

no  UAV 

1  UAV 

1  UAV 

2  UAVs 

2  UAVs 

no  ships/  2UAVs 

no  ships/  2UAVs 

Red  Tactics 

Approach  merchant 

vessels 

Approach  shoreline 

2.  Design  Factors 

In  every  experiment,  it  is  impossible  to  account  for  every  factor  that  might 
affect  the  outcome.  Investigating  too  many  factors  would  make  the  analysis 
extremely  difficult,  whereas  having  too  few  factors  might  result  in  inaccurate 
conclusions.  This  study  varies  21  factors  (Table  12)  that  seem  to  have  the  most 
important  impact  on  the  performance  of  the  surveillance  system.  The  factors 
were  chosen  based  on  opinions  of  subject  matter  experts,  as  well  as  the  author’s 
experience.  Although  in  simulation  experiments  all  the  factors  are  manipulated,  in 
reality  this  is  not  the  case.  For  this  reason,  the  factors  are  distinguished  into 
controllable  and  uncontrollable  (Sanchez  &  Wan,  2012). 
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Table  12. 


Independent  factors  varied  in  the  DOE. 


Platform 

Variable 

Type 

Unit 

Range 

1. 

Speed 

Continuous 
(sampled  in 
discrete  values) 

knots 

70-150 

2. 

Max  Detection  Range 

Continuous 

meters 

5000- 

30000 

3. 

UAV 

Avg.  Time  Between 
Detections  (Max  range) 

Continuous 

seconds 

5-120 

4. 

Probability  of  Classification 
at  Min  Range 

Continuous 

0.7 

1 

5. 

Communication  Latency 

Continuous 

seconds 

30-120 

6. 

Communication  Accuracy 

Continuous 

percentage 

50-100 

7. 

Speed 

Continuous 
(sampled  in 
discrete  values) 

knots 

15-25 

8. 

Blue 

Ships 

Max  Detection  Range 

Continuous 

meters 

5000- 

25000 

9. 

Avg.  Time  Between 
Detections  (Max  range) 

Continuous 

seconds 

8-  240 

10. 

Probability  of  Classification 
at  Min  Range 

Continuous 

0.7 

1 

11. 

Blue 

Ground 

Radars 

Max  Detection  Range 

Continuous 

meters 

10000- 

50000 

12. 

Avg.  Time  Between 
Detections  (Max  range) 

Continuous 

seconds 

5-120 

13. 

Probability  of  Classification 
at  Min  Range 

Continuous 

0.7 

1 

14. 

Opera¬ 

tional 

Centre 

Communication  Latency 

Continuous 

seconds 

30-120 

15. 

Communication  Accuracy 

Continuous 

percentage 

50-100 

16. 

Stealth 

Continuous 

percentage 

30-90 

17. 

Red  Ships 

Detection  Range 

Continuous 

meters 

5000- 

25000 

18. 

Speed 

Continuous 
(sampled  in 
discrete  values) 

knots 

15-35 

19. 

Fishing 

Vessels 

#  of  fishing  vessels 

Discrete 

- 

5-50 

20. 

Merchant 

vessels 

#  of  merchant  vessels 

Discrete 

- 

4-  30 

21. 

speed 

Continuous 
(sampled  in 
discrete  values) 

knots 

8-20 
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The  controllable  factors  are  those  that  are  varied  at  will  by  the 
experimenter.  All  the  variables  that  deal  with  the  Blue  sensors  (Blue  ships,  UAVs, 
ground  radars  and  Operational  Centre)  fall  in  this  category.  These  variables 
describe  the  detection  capabilities  of  the  Blue  sensors  and  possibly  have  an 
effect,  major  or  minor,  to  the  response  variable.  These  factors  include  the 
detection  and  classification  ranges,  the  probabilities  of  detection,  speed  and 
communication  capabilities. 

Uncontrollable  or  noise  factors  are  those  that  cannot  be  manipulated  by 
the  decision  maker  in  reality.  In  this  experiment,  the  number  of  merchant  or 
fishing  vessels,  their  speed  and  the  configurations  of  the  Red  ships  fall  into  this 
category.  They  may  or  may  not  have  a  significant  effect  on  the  outcome,  but  by 
varying  within  valid  ranges,  they  introduce  a  natural  variability  in  the  models.  In 
that  way  more  accurate  conclusions  are  drawn. 

In  addition  to  the  21  independent  variables  that  we  explicitly  vary  in  the 
DOE,  there  are  an  additional  52  variables  that  depend  on  one  of  the 
aforementioned  factors.  These  variables  describe  the  sensors’  capabilities,  and 
this  technique  is  extremely  useful  when  the  Advanced  Detection  Mode  is  used 
(see  Chapter  III). 

The  Maximum  Classification  Range  is  set  to  0.8*  Maximum  Detection 
Range  for  all  the  agents.  Moreover,  all  the  Blue  units  use  the  Advanced 
Detection  Mode  where  a  range  table  can  be  set  up  to  define  how  the  sensing 
capability  falls  off  with  increased  distance.  Therefore,  their  detecting  ranges  are 
divided  into  five  homocentric  cycles,  each  one  with  a  different  chance  to  detect 
(or  classify)  a  target.  The  “upper”  range  value  for  each  of  the  four  inner  cycles  is 
calculated  by  multiplying  the  maximum  detection  (or  classification)  range  by  1/5, 
2/5,  3/5  and  4/5,  respectively.  For  example,  if  the  maximum  detection  range  is 
25,000  meters,  Zone  1  will  be  up  to  5,000  meters,  Zone  2  from  5,000  to  10,000 
meters,  etc. 
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For  the  detection,  those  “zones”  are  associated  with  Average  Times 
between  Detections.  As  indicated  in  the  previous  paragraph,  these  times  are 
calculated  by  multiplying  the  Average  Times  between  Detections  for  the 
maximum  detection  (or  classification)  range  by  1/5,  2/5,  3/5  and  4/5,  respectively. 

For  the  classification,  the  five  ranges  are  assigned  with  probabilities  of 
classification.  In  this  case,  taking  as  a  reference  the  first  zone,  which  has  the 
maximum  probability  of  classification,  we  calculate  the  rest  by  multiplying 
successively  by  0.9,  0.8,  0.7  and  0.6.  In  that  way,  at  the  verge  of  the 
classification  range  we  obtain  60  percent  of  the  nominal  maximum  probability  of 
classification.  All  the  dependent  variables  are  depicted  in  Table  13. 
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Table  13. 


Dependent  variables  in  the  DOE 


Dependent  Variable 

Associated 
Independent  Variable 

Multiplying  by 

1. 

UAV 

Maximum  Classification  Range 
(Zone  5) 

Maximum  Detection 
Range  (Zone  5) 

0.8 

2. 

1st  detection  range  (Zone  1) 

Maximum  Detection 
Range  (Zone  5) 

0.2 

3. 

2nd  detection  range  (Zone  2) 

0.4 

4. 

3rd  detection  range  (Zone  3) 

0.6 

5. 

4th  detection  range  (Zone  4) 

0.8 

6. 

Time  Between  Detections  (Zone  1) 

Time  Between 
Detections  IN 
Maximum  Range 
(Zone  5) 

0.2 

7. 

Time  Between  Detections  (Zone  2) 

0.4 

8. 

Time  Between  Detections  (Zone  3) 

0.6 

9. 

Time  Between  Detections  (Zone  4) 

0.8 

10. 

1st  classification  range  (Zone  1) 

Maximum 

Classification  Range 
(Zone  5)* 

(*  it  is  also  dependent) 

0.2 

11. 

2nd  classification  range  (Zone  2) 

0.4 

12. 

3rd  classification  range  (Zone  3) 

0.6 

13. 

4th  classification  range  (Zone  4) 

0.8 

14. 

Probability  of  Classification  (Zone  2) 

Probability  of 
Classification  at  Min 
Range (Zone  1) 

0.9 

15. 

Probability  of  Classification  (Zone  3) 

0.8 

16. 

Probability  of  Classification  (Zone  4) 

0.7 

17. 

Probability  of  Classification  (Zone  5) 

0.6 

18. 

Blue 

Ships 

Maximum  Classification  Range 
(Zone  5) 

Maximum  Detection 
Range  (Zone  5) 

0.8 

19. 

1st  detection  range  (Zone  1) 

Maximum  Detection 
Range  (Zone  5) 

0.2 

20. 

2nd  detection  range  (Zone  2) 

0.4 

21. 

3rd  detection  range  (Zone  3) 

0.6 

22. 

4th  detection  range  (Zone  4) 

0.8 

23. 

Time  Between  Detections  (Zone  1) 

Time  Between 
Detections  IN 
Maximum  Range 
(Zone  5) 

0.2 

24. 

Time  Between  Detections  (Zone  2) 

0.4 

25. 

Time  Between  Detections  (Zone  3) 

0.6 

26. 

Time  Between  Detections  (Zone  4) 

0.8 

27. 

1st  classification  range  (Zone  1) 

Maximum 

Classification  Range 
(Zone  5)* 

(*  it  is  also  dependent) 

0.2 

28. 

2nd  classification  range  (Zone  2) 

0.4 

29. 

3rd  classification  range  (Zone  3) 

0.6 

30. 

4th  classification  range  (Zone  4) 

0.8 

31. 

Probability  of  Classification  (Zone  2) 

Probability  of 
Classification  at  Min 
Range (Zone  1) 

0.9 

32. 

Probability  of  Classification  (Zone  3) 

0.8 

33. 

Probability  of  Classification  (Zone  4) 

0.7 

34. 

Probability  of  Classification  (Zone 

5) 

0.6 

35. 

Ground 

Radars 

Maximum  Classification  Range 
(Zone  5) 

Maximum  Detection 
Range  (Zone  5) 

0.8 

36. 

1st  detection  range  (Zone  1) 

Maximum  Detection 
Range  (Zone  5) 

0.2 

37. 

2nd  detection  range  (Zone  2) 

0.4 
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38. 

3rd  detection  range  (Zone  3) 

0.6 

39. 

4th  detection  range  (Zone  4) 

0.8 

40. 

Time  Between  Detections  (Zone  1) 

Time  Between 
Detections  IN 
Maximum  Range 
(Zone  5) 

0.2 

41. 

Time  Between  Detections  (Zone  2) 

0.4 

42. 

Time  Between  Detections  (Zone  3) 

0.6 

43. 

Time  Between  Detections  (Zone  4) 

0.8 

44. 

1st  classification  range  (Zone  1) 

Maximum 

Classification  Range 
(Zone  5)* 

(*  it  is  also  dependent) 

0.2 

45. 

2nd  classification  range  (Zone  2) 

0.4 

46. 

3rd  classification  range  (Zone  3) 

0.6 

47. 

4th  classification  range  (Zone  4) 

0.8 

48. 

Probability  of  Classification  (Zone  2) 

Probability  of 
Classification  at  Min 
Range (Zone  1) 

0.9 

49. 

Probability  of  Classification  (Zone  3) 

0.8 

50. 

Probability  of  Classification  (Zone  4) 

0.7 

51. 

Probability  of  Classification  (Zone  5) 

0.6 

52. 

Red 

Ships 

Maximum  Classification  Range 

Maximum  Detection 
Range 

0.8 

B.  NEARLY  ORTHOGONAL  LATIN  HYPERQUBE  DESIGNS 

Latin  Hypercubes  (LH)  have  been  increasingly  popular  in  the  designs  of 
computer  experiments  because  they  provide  a  flexible  way  of  constructing 
efficient  designs  to  explore  quantitative  factors.  They  have  “some  of  the  space¬ 
filling  properties  of  factorial  designs  with  fine  grids  but  require  orders  of 
magnitude  less  sampling”  (Sanchez  &  Wan,  2011).  If  we  had  a  small  number  of 
factors,  we  could  create  a  full  factorial  design  to  study  all  possible  combinations 
of  the  factor  levels.  However,  this  is  impossible  when  we  deal  with  a  large 
number  of  input  variables  because  the  time  needed  would  increase 
geometrically. 

Nearly  Orthogonal  Latin  Hypercubes  (NOLH)  are  “LH  which  have  good 
space-filling  and  orthogonality  properties  for  a  small  or  moderate  number  of 
factors.  These  designs  are  not  square,  but  the  number  of  design  points  is 
radically  fewer  than  the  numbers  for  the  gridded  designs”  (Sanchez  &  Wan, 
2011). 


Cioppa  and  Lucas  (2007,  pp.  45-55)  presented  NOLH  with  good  space¬ 
filling  properties  for  up  to  22  factors  in  as  few  as  129  input  combinations,  and 
later  up  to  29  factors  in  257  design  points.  An  Excel  spreadsheet,  developed 
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from  the  Simulation  Experiments  and  Efficient  Design  (SEED)  Center  for  Data 
Farming  at  the  Naval  Postgraduate  School  in  Monterey,  is  used  to  create  those 
specific  combinations  that  the  NOLH  design  demands,  as  shown  in  Figure  26. 


Figure  26.  Snapshot  of  the  spreadsheet  used  for  the  DOE. 


There  are  three  major  benefits  of  NOLH  in  this  study: 

•  Only  257  design  points  are  necessary  in  this  DOE.  Instead,  if  we 
wanted  to  explore  the  23  factors  as  2-level,  and  not  as  continuous, 
we  would  need  more  than  eight  million  design  points. 

•  It  is  a  good  space-filling  design  because  “the  design  points  are 
scattered  throughout  the  experimental  region  with  minimal 
unsampled  regions”  (Cioppa  &  Lucas,  2007).  Figure  27  is  a 
scatterplot  matrix  that  depicts  the  projections  of  the  full  design  onto 
each  pair  of  factors.  It  can  be  easily  seen  that  this  property  holds. 


74 


1  110 

2  » 

3  70j 

I  £  25000 

3  *  5000 

a  : 

S  o 

§  09 

V  08 

3  or  j 

If  ” 

i5  " 

Is  90 

u  2  70 

Is  SO 

1!  * 

%$!  22 

S  J  <8; 

1®  1S000 

5^  •  5000 

Ij  » 

III  ? 

« 

Ss  08J 

®  07- 

?!  ■» 

II  : 

/ 

P 

» 

K 

8f 

m 

ii 

m 

8 

i'iMSs- 

8 

1 

m 

mu 

Si 

1 

K 

» 

SI 

B 

Sill 

4se«g 

P 

7 

p 

HP 

ill 

m 

ii 

i*-li  fri. 

m 

St 

M 

H 

s 

ill 

m 

S 

8 

mas 

1  i"!  |- 

i 

ffi 

/ 

8 

Ii 

m 

M 

m 

|| 

PI 

m 

1 

sit 

n 

m 

ii 

mg 

S 

:#s 

SM 

w$ 

ll 

n 

®§ 

i 

9 

■V  r.  <* 

i 

■ 

&& 

m> 

IMS 

W?; 

;Swv& 

w^Sr 

Wx 

m 

mr; 

M 

im. 

til 

® 

H 

•a?- 

ijj:-i:i!E 

-•n:iI:-:-*' 

m 

ip 

§| 

m 

/ 

Wl. 

11 

H 

11 

m 

r>#' 

m& 

W; 

iK?l 

-,r. .j- 

m 

M 

m 

m 

15 

il 

HI 

§§ 

11 

m 

/ 

«5ji* 

n 

- 

8 

m 

0m 

M 

m 

S 

H 

m. 

H 

il 

m 

’♦'J'-  .V  , 

fasr.yS 

i® 

vr 

‘.w«W- 

TOifS 

sf'jUr* 

'I^uyvi 

■ 

j?£y§i 

‘XVK- 

"r-  "  ." 

@i 

‘VttT 

*ry!‘JMZ 

S§ 

HI 

M 

Ii®:? 

? 

H 

ffi 

p 

'M 

m 

:&ia 

Me 

1 

m 

m 

m 

jSllEi 

iSrTS 

p 

ilf 

* 

H 

M 

m 

if 

*w5 

77 

tm 

sfe? 

/ 

■ 

m 

ii 

m 

PI 

m 

if 

IP 

m 

m 

M* 

Si 

'  n'V- 

|iS; 

m 

HH 

in 

nil 

PI 

si§ 

m 

-  '  - 

/ 

m 

feiSv 

PI 

fern 

w& 

zm >• 

M 

SIB 

m 

4tfP2 

$m 

w§ 

8^ 

■$$$ 

«!g5$ 

2»iiV 

«S 

pli 

M 

M 

IS 

pfcffe 

m 

mt 

/ 

8 

ii 

M 

H 

mg 

mse 

w& 

i 

w 

m 

ip 

O  0 

S"  90 

ft,  : 

a  I  40X0 

if  20000 
,0000 

<1  80 
it.  : 
if  :: 
ll 

If,  « 

0  5  20 

•  2  90 

Si  '• 

see  so- 

1 

I  70 

s  50 

S  30 

2ft 

H 

5000' 

!  - 

I?  » 
fi  ” 
ii  » 

ft  T 
f!  : 

§§ 

m 

m 

&rr£ 

m 

m. 

gg 

®§ 

/ 

m 

P 

m 

mi. 

m 

*4s 

S 

sill? 

m 

m 

§l$f 

m 

ill 

m 

vM 

H5'5s 

M 

SB 

is® 

89 

5S& 

si 

/ 

89 

s 

P 

'■ 

>v4 

♦sit 

M 

S;ilii;!s 

:■'■■ 

M 

M 

§§ 

M 

s 

11 

K 

K 

m 

/ 

pm 

tern 

'£m 

»■ 

■PI 

»42f 

sm 

*>  v- 

W-; 

i;»!,J:  i;:;> 

:!s'  •!:•'! 

d& 

m 

$S3r 

s 

M 

pi 

Siisp’: 

US 

H 

STW 

Ml 

«5W. 

8 

B 

M 

/ 

a3a< 

-  . 

P 

£*<% 

mi 

PI 

B 

:i.  ' 

M 

fH 

n 

aWfe 

ps 

fes 

8 

n 

8 

m 

?>£& 

/ 

m 

fMjfc* 

ms 

ii 

m 

8 

M 

!?Sa§ 

m 

IS 

llg 

. . 

r<<? 

ii 

s 

f£ 

>’- 

m 

m 

m. 

/ 

Ml 

m 

■^i 

P 

PM 

ii;::i!t; 

m 

H 

SSg 

m 

ii 

m 

tyv-jt 

n 

8 

8 

M 

n 

/ 

M 

M 

ii 

m 

s 

■ 

m 

81 

8 

m 

,  :r 

Irff:. 

g 

m 

M 

* 

gfigci 

m 

-3*f 

B 

/ 

a 

m 

,  i».!  !i,;*; 

M 

» 

1 

m 

r>^, 

m 

W{5,s 

m 

m 

it 

SM 

s 

’  - 

g 

S 

M 

/ 

s 

SSflSl 

tsfed 

M 

|§g 

H 

m 

w 

f£B 

5;gr^f 

m 

M 

a 

W 

fi 

8S 

M 

7 

e 

ii 

M 

* 

KXijB 

z5ff*e 

Ms# 

5^5 

jsas 

m 

t® 

fl 

fe: 

i$ 

m 

11 

B 

s 

/ 

in 

ms 

§££|j 

iMf. 

lii&L 

pglf 

SSS: 

'i5-r5t 

WiM 

: 

Wi& 

HI 

-7:4 

:7-.; 

PH 

'090  <20  S00025000  070  60  0  70809  20  60  100  50  70  90  14182226  600070000  070  60100  0  70  80  9  20  60  100  SO  70  90  9000706S?  020  60  0  7  0  8  0  9  20  60  100  SO  70  90  >0  SO  70  S00070000  1S207SM  010  10  S10  70  810  14  18 

UAV,Sp**0'_M»*t>**U4»sTim»©«.  MV_PrCI*»  ,CoroTW.a  _Comm»Am«S'X'»,St«upi  «_AvflTHn»l*Sf*c*,PiC  ic*,Comfniei_Co»nm».»fJA*iO*IR  A^Tlrr«»8«  04»_Pre>oa.  »«_Commi  t«_Co«nnu  RM_S4*»mi»a,04tRani  R»a_$©««d  nt>«rfiiriV«i  6*M*«0iV(  *v«t,«s_S 


Figure  27.  Scatterplot  Matrix  for  all  the  factors  in  the  DOE. 

•  Near  orthogonality  ensures  that  all  correlations  between  the  factors 
are  very  small,  between  the  interval  (-0.03,  0.03).  Figure  28 
confirms  this  property  for  the  first  ten  factors  because  the  maximum 
absolute  pairwise  correlation  is  0.0096. 
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Figure  28.  Snapshot  of  correlation  matrix  for  the  first  ten  factors  of  the  DOE. 


C.  EXPERIMENT  EXECUTION 

Our  simulation  experiment  was  run  on  a  high-performance  cluster  with  44 
processors,  called  “Reaper.”  The  software  packages  xStudy,  OldMcData  and 
Condor  were  used  in  this  study  to  generate  the  data  files.  XStudy,  written  by 
SEED  Center  Research  Associate  Steve  Upton,  is  a  graphical  user  interface  for 
generating  a  study  design  file  in  .xml  format.  Among  other  information,  it 
identifies  the  model  used,  the  number  of  replications,  initial  random  seeds  and 
what  variables  will  be  used  for  variation.  It  is  also  used  to  map  each  factor  of  the 
DOE  to  a  specific  element  of  the  MANA  scenario  file. 

OldMcData  (OMD)  is  a  software  application  designed  to  do  data  farming 
runs,  from  running  large  simulation  experiments  on  a  distributed  computer  cluster 
to  multiple  replications  of  a  single  excursion  on  a  single  machine.  For  runs  on  a 
distributed  computing  cluster,  OMD  uses  Condor,  an  open-source  distributed 
computing  environment,  to  handle  the  scheduling  and  managing  of  the  active 
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jobs.  All  the  software  mentioned  in  the  previous  paragraphs  is  available  through 
the  NPS  SEED  Centre’s  website  (http://harvest.nps.edu/). 


77 


THIS  PAGE  INTENTIONALLY  LEFT  BLANK 


78 


V.  RESULTS  AND  DATA  ANALYSIS 


A.  MEASURES  OF  EFFECTIVENESS  AND  DATA  PREPARATION 

This  section  describes  the  calculation  made  to  obtain  the  two  MOEs  and 
the  reasons  that  led  us  to  manipulate  the  data  taken  for  the  1-UAV  case. 
Additionally,  this  section  contains  our  study  about  the  necessary  assumptions 
that  have  to  be  met  to  conduct  parametric  statistical  methods. 

1.  Measures  of  Effectiveness 

As  illustrated  in  Chapter  III,  two  Measures  of  Effectiveness  (MOE)  are 
used.  The  first  is  “The  proportion  of  time  the  enemy  ships  are  positively 
identified.”  This  MOE,  referred  to  as  MOE  1,  estimates  how  the  surveillance 
system  of  the  Blue  Force  performs.  This  measure  is  indirectly  calculated  using 
the  amount  of  “Fuel”  of  each  Red  Agent.  At  the  beginning  of  a  run,  each  one  of 
the  four  Red  ships  is  equipped  with  a  certain  amount  of  fuel  that  is  set  to  remain 
constant  any  time,  unless  it  is  identified  by  the  Blue  Force.  For  the  time  period 
the  targets  are  monitored,  they  consume  one  unit  of  fuel  per  time  step.  At  the  end 
of  each  simulation  run,  the  fuel  that  all  the  Red  Ships  have  consumed  is  summed 
up  and  divided  by  the  number  of  Red  Ships  and  the  Total  Run  Time.  In  both  the 
scenarios,  there  are  four  Red  Ships  that  try  to  reach  their  goal  without  being 
detected.  Finally,  the  result  of  MOE  1  is  a  number  expressed  as  a  percentage. 
Formula  (1 )  describes  the  calculation  of  MOE  1 : 

Sum  of  Fuel  Consumed  by  Re  d  Ships 

MOE  1  = - - - - - —  m 

A*  Total  RunTime  '  ' 

The  second  MOE,  referred  to  as  MOE  2,  is  “The  proportion  of  time  the 
merchant  vessels  are  positively  identified.”  Although  we  are  mostly  interested  in 
the  surveillance  performance  over  the  Red  Ships,  this  metric  is  used  as  an 
element  of  force  validation  to  the  simulation  model.  It  is  expected  that  the  second 
MOE  will  be  higher  in  all  cases  because  Merchant  Vessels  do  not  use  any  tactics 
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to  avoid  detection  from  the  Blue  Force.  Additionally,  it  will  be  interesting  to  see  to 
what  extent  this  metric  is  affected  by  the  various  combinations  of  the  Blue 
Sensors. 

The  same  procedure  for  MOE  1  is  used  to  calculate  MOE  2.  Like  Red 
Ships,  Merchant  Vessels  consume  “fuel”  for  the  time  period  they  are  monitored 
by  the  surveillance  system.  The  overall  amount  of  fuel  that  the  Merchant  Vessels 
consumed  is  divided  by  the  aggregated  time  they  were  in  the  simulation  run. 
Formula  (2)  describes  the  calculation  of  MOE  2: 

Sum  of  Fuel  Consumed  byMerchant  Vessels 

MOE  2  = - - - - - 

k  *  Total  Run  Time  ^ 

where  k  is  the  number  of  Merchant  Vessels  in  the  model. 

The  “Open  Sea”  model  involves  16  Merchant  Vessels,  and  the  “Many 
Islands”  model  involves  10.  Again,  the  result  yields  a  percentage. 

2.  Data  Preparation 

Concerning  the  case  when  only  one  UAV  is  deployed,  the  preliminary  runs 
showed  that  the  area  from  where  it  is  launched  affects  the  performance  of  the 
UAV,  with  respect  to  the  MOEs,  and  consequently,  of  the  whole  surveillance 
system.  When  two  UAVs  operate,  the  AOI  is  equally  divided  in  two  sections  and 
each  UAV  is  assigned  in  a  designated  section.  However,  when  only  one  is 
available,  it  covers  all  the  AOI  by  itself.  According  to  the  attributes  that  have  been 
given  to  the  UAV,  it  patrols  along  a  particular  route  until  it  locates  an  unknown 
contact.  Then  it  heads  towards  it  to  identify  it.  As  it  is  easily  understood,  the  more 
contacts  there  are  close  to  the  launch  site,  the  more  the  UAV  contributes  to  the 
surveillance  system.  That  is,  if  the  launch  site  is  close  to  areas  with  traffic,  the 
UAV  will  have  more  opportunities  to  locate  the  targets.  Additionally,  the  targets 
will  be  identified  earlier.  The  difference  in  the  MOE  1  with  respect  to  the  location 
of  a  single  UAV  launched  for  the  “Open  Sea”  Model  is  shown  in  Figure  29.  When 
the  UAV  started  its  deployment  from  the  Northwest,  the  Surveillance  System 
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detected  the  Red  Ships  on  average  60.7%  of  the  time  (SD=O.057).  In  contrast, 
when  the  UAV  started  its  patrol  from  Southeast,  it  scored  56.9%  (SD=0.056).  An 
independent  sample  f-test  with  a  two  tailed  alpha  level  of  .05  indicated  that 
scores  were  significantly  higher  for  the  first  case  than  the  second  (f(998)=  -9.381, 
pcO.OOOl).  The  same  issue  also  occurred  in  the  “Many  Islands”  model 
(f(998)=  -5.244,  p<0.001).  More  details  are  provided  in  Appendix  A. 


4)  ▼  Distributions  Case=Open Sea-1  UAV  (North) 
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41  ~  Distributions  Case=Open  Sea-1  UAV(  South) 
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Figure  29.  Summary  statistics  for  when  a  single  UAV  is  launched  from  Northwest 

or  Southeast  in  the  “Open  Sea”  model. 


In  order  to  mitigate  the  dissimilarities  in  geography  and  target 
arrangements,  two  different  scenarios  were  run  every  time  one  UAV  is  needed. 
In  the  first  scenario,  the  UAV  is  launched  from  a  location  in  the  Northwest, 
whereas  in  the  second,  the  UAV  started  its  deployment  from  the  Southeast 
(“Open  Sea”  model)  or  the  Southwest  (“Many  Islands”  model),  respectively. 
When  all  the  data  had  been  collected,  we  calculated  the  averages  of  the  MOEs 
of  the  two  1-UAV  scenarios.  In  that  way,  it  can  be  considered  that  the  UAV  was 
launched  from  a  location  in  the  middle  of  the  true  launch  sites. 
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3.  Assumptions  for  Statistical  Analysis  Method 

Our  decision  whether  to  use  parametric  or  non-parametric  method  for  the 
analysis  is  based  on  whether  the  assumptions  of  independence,  equality  of 
variance,  and  normality  are  met.  Below  we  study  those  assumptions  for  the 
“Open  Sea”  model. 

•  Independence  of  replications:  The  observations  within  each 
treatment  group  must  be  independent  of  one  another.  Our 
experiment  was  completely  randomized  because  we  derived  our 
data  through  a  stochastic  model.  MANA  uses  a  random  number 
generator  to  randomize  the  starting  positions  of  agents  within  their 
defined  home  boxes  as  well  as  the  probabilities  of  detection. 
Consequently,  scenarios  can  have  considerably  different  outcomes 
each  time  they  are  run.  Therefore,  the  assumption  of  independence 
holds. 

•  Equality  of  Variance:  The  treatments  must  have  approximately 
equal  variances.  The  side-by-side  boxplots  for  the  four  cases  we 
are  interested  in  for  the  “Open  Sea”  Model  are  shown  in  Figure  30. 
It  can  be  seen  that  the  variance  is  approximately  the  same  across 
the  different  cases.  Additionally,  because  we  have  a  large  sample 
size,  we  can  assume  that  the  equal  variance  assumption  holds. 
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Figure  30.  Side-by-side  boxplots  for  the  four  scenarios  in  the  “Open  Sea”  model. 


•  Normal  Population:  The  data  of  each  treatment  must  come  from 
populations  that  are  Normal  or  close  to  Normal.  The  histograms  of 
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Residuals  for  the  four  cases  are  shown  in  Figure  31 .  Although  the 
histogram  is  skewed  on  the  left  in  the  case  where  there  are  no 
UAVs,  it  can  be  considered  that  all  the  cases  adequately  meet  the 
Near  Normality  Assumption. 


Figure  31 .  Histogram  of  the  residuals  across  the  four  cases  in  the  “Open  Sea” 

model. 

The  same  assumptions  also  hold  for  MOE  2  as  well  as  for  both  MOEs  in 
the  "Many  Islands"  model,  as  presented  in  Appendix  B.  In  addition,  Appendix  C 
contains  the  study  for  the  assumptions  for  both  the  MOEs  for  the  “Many  Islands” 
Model.  Finally,  a  two-tailed  alpha  level  of  .05  was  employed  for  all  statistical 
tests. 

B.  RESULTS  OF  INITIAL  RUNS 

This  section  studies  the  results  of  500  initial  runs  that  were  conducted  for 
each  one  of  the  “Open  Sea”  and  the  “Many  Islands”  models.  These  runs  used  a 
particular  configuration  of  the  capabilities  of  the  agents  involved  in  the  models. 
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The  results  from  the  simulation  were  used  to  compare  the  performance  of  the 
surveillance  system  in  the  different  scenarios  with  respect  to  the  MOEs  defined 
earlier.  Descriptive  statistics  of  the  initial  runs  are  presented  and  hypotheses  for 
the  research  questions  are  tested  using  appropriate  statistical  methods.  Results 
from  the  "Open  Sea"  model  are  presented  first. 

1.  “Open  Sea”  Model 

a.  Descriptive  Statistics 

The  distributions  of  the  percentage  of  Red  Ships  that  had  been 
classified  across  the  four  different  cases  are  shown  in  Figure  32.  We  notice  that 
on  average,  the  surveillance  system  yielded  the  highest  numbers  of  detections 
when  two  UAVs  operate  (Mean=  0.668,  SD=0.052).  In  contrast,  the  current 
surveillance  system  with  no  UAVs  performed  the  worst  (Mean=  0.419, 
SD=0.049). 
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Figure  32.  Distributions  of  MOE  1  for  the  500  initial  runs  across  the  four  cases  in 

the  “Open  Sea”  model. 


'The  distributions  of  the  percentage  of  Merchant  Vessels  that  had 

been  classified  across  the  four  different  cases  are  shown  in  Figure  33.  As 
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expected,  on  average  the  surveillance  system  yielded  the  highest  numbers  of 
detections  when  two  UAVs  operate  (Mean=  0.791,  SD=0.033).  In  contrast,  the 
current  surveillance  system  with  no  UAVs  performed  the  worst  (Mean=  0.477, 
SD=0.036). 
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Figure  33.  Distributions  of  MOE  2  for  the  500  initial  runs  across  the  four  cases  in 

the  “Open  Sea”  model. 


The  average  percentage  of  time  the  targets  are  identified  is  shown 
in  Figure  34.  Concerning  the  Red  Ships,  besides  the  best  and  worst  cases  which 
were  commented  on  earlier,  the  1-UAV  case  (which  concludes  also  two  Blue 
Ships)  performs  slightly  better  than  the  case  where  only  two  UAVs  operate 
without  any  surface  support. 
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Figure  34.  Average  percentage  of  time  the  targets  are  detected  across  the  four 
cases  of  the  “Open  Sea”  model.  (Standard  Error  ranges  from  0.0013 
to  0.0027  and  would  not  be  easily  seen  on  the  bars). 


It  is  worth  noting  that  the  Merchant  Vessels  results  are  consistent 
with  those  of  the  Red  Ships.  Additionally,  in  every  case  the  Red  Ships  succeed  in 
being  detected  fewer  times  than  the  Merchant  Vessels  using  specific  tactics  to 
avoid  the  adversary  sensors.  This  provides  an  element  of  force  validation  to  the 
simulation  model. 

b.  Hypotheses  Testing 

One  of  the  primary  questions  of  this  thesis  is  whether  UAVs  are 
able  to  mitigate  the  incapability  of  the  current  surveillance  system  to  detect 
targets  in  complex  maritime  environments,  given  that  the  target's  tactic  is  to 
approach  transiting  merchant  vessels.  In  other  words,  does  adding  UAVs  to  a 
surveillance  system  that  uses  traditional  units,  such  as  ships  and  radars,  help  it 
become  more  efficient?  The  hypothesis  tested  is: 

•  H0:  there  is  no  difference  in  the  surveillance  system 

performance  whether  it  uses  UAVs  or  not,  given  that  the 
target’s  tactic  is  to  approach  transiting  merchant  vessels 
(Mi=M2=M3=M4  where  the  four  populations  refer  to  the  four 
different  cases). 
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•  Ha:  a  surveillance  system  with  UAVs  performs  better  than 
without  UAVs,  given  that  the  target’s  tactic  is  to  approach 
transiting  merchant  vessels  (pi^p2^P3^M4  where  the  four 
populations  refer  to  the  four  different  cases). 

To  test  the  hypothesis  above,  Analysis  of  Variance  (ANOVA)  was 
used.  This  parametric  method  demands  particular  assumptions,  such  as 
independence  of  groups,  the  equal  variance  assumption  and  the  Normal 
Population  assumption  (De  Veaux  et.  al,  2009).  In  the  previous  section,  we 
proved  that  all  these  assumptions  were  met  for  both  the  MOEs. 

The  ANOVA  (R2= 0.756)  showed  that  the  null  hypothesis  was 
rejected:  there  was  a  significant  difference  between  the  four  cases  (F( 3, 
1996)=2061 .5,  p  <  0.001)  (Figure  35).  That  is,  the  number  of  UAVs  has  a 
significant  effect  on  the  performance  of  the  surveillance  system. 
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Figure  35.  ANOVA  for  the  initial  runs  of  the  “Open  Sea”  model. 


To  uncover  which  UAV  cases  were  significantly  different  from  each 
other,  two  sample  f-tests  were  conducted  in  terms  of  the  mean  proportion  of  time 
Red  Ships  were  positively  identified.  All  the  pairwise  differences  after  conducting 
Student’s  f-test  are  shown  in  Table  14.  We  notice  that,  statistically,  every  case 
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differs  from  each  other.  Furthermore,  the  largest  “improvement”  occurs  when  we 
go  from  the  current  system  (no-UAV)  to  a  solution  with  one  or  two  UAVs. 

Table  14.  Ordered  pairwise  differences  report  after  Student’s  f-test  (“Open 

Sea”). 


^Ordered  Differences  Report 

Level  -Level  Difference  StdErrDif  Lower  CL  Upper  CL  p-Value 

0penSea-2UAVs  OpenSea-noUAV  0.2493570  0.0032650  0.2429539  0.2557601  <.0001* 

OpenSea-lUAV  OpenSea-noUAV  0.1671629  0.0032650  0.1607598  0.1735660  <0001' 

0penSea-2UAVs  OpenSea-noBlueShips  0.1404468  0.0032650  0.1340436  0.1468499  <0001* 

OpenSea-noBlueShips  OpenSea-noUAV  0.1089103  0.0032650  0.1025071  0.1153134  <0001* 

0penSea-2UAVs  OpenSea-lUAV  0.0821941  0.0032650  0.0757910  0.0885973  <0001* 

OpenSea-lUAV  OpenSea-noBlueShips  0.0582526  0.0032650  0.0518495  0.0646558  <0001' 

2.  “Many  islands”  Model 

a.  Descriptive  Statistics 

The  distributions  of  the  percentage  of  Red  Ships  that  had  been 
classified  across  the  four  different  cases  are  shown  in  Figure  36.  We  notice  that 
on  average  the  surveillance  system  yielded  the  highest  numbers  of  detections 
when  two  UAVs  operate  (Mean=  0.775,  SD=0.036).  In  contrast,  the  current 
surveillance  system  with  no  UAVs  performed  the  worst  (Mean=  0.480, 
SD=0.062). 
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Figure  36.  Distributions  for  MOE  1  for  the  500  initial  runs  across  the  four  cases  in 

the  “Many  Islands”  model. 

The  distributions  of  the  percentage  of  Merchant  Vessels  that  had 
been  classified  across  the  four  different  cases  are  shown  in  Figure  37.  As 
expected,  on  average  the  surveillance  system  yielded  the  highest  numbers  of 
detections  when  two  UAVs  operate  (Mean=  0.914,  SD=0.028).  In  contrast,  the 
current  surveillance  system  with  no  UAVs  performed  the  worst  (Mean=  0.780, 
SD=0.078). 


-i  •  Distributions Case«Manylslands-1UAV 
A  -  %MerchantClassified 


•  H - □□ — 1 

1-rlT 

L 

A  Quantiles 

1000%  manmum  0  04*07 


00  5% 
975% 

90  0% 

75  0% 

50  0% 
250% 
100% 
25% 
05% 
00% 

-  Distributions  Case*Manylslands-2UAVs 
3  *  V.MerchantC  lassified 


065  07  075  08  085  09  005 


0  04132 
092098 
0  90106 
quamie  0  87748 
median  0  85219 
quamie  0  81816 
0  78601 
0  74001 
0  71618 
minimum  0  63736 


-  Summary  Statistics 
Mean  0  8459067 

StdDev  0  0453674 

SM  Eff  Mean  0  0020289 
Upper  95%  Mean  0  8498929 
Lower  95%  Uean  0  8419205 


N 


500 


■*  !a|  i  m  i  I 

Quantiles  -  *  Summary  Statistics 

1000%  manmum  0  97727  Mean  0  9144857 

■  -1 - CTH 

1 1 .  rn-fflf  - 

lw 

97  5%  0  96219  SM  Err  Mean  0  0012623 

90  0%  0  94747  Upper  95%  Mean  0  9169657 

75  0%  quamie  0  93367  Lower  95%  Mean  0  9120057 

500%  median  0  91877  N  500 

25  0%  quamie  08991 

,  jfl 

1 

062  '  086  '  09 

094  09C 

100%  0  87586 

25%  0  84801 

n  Tm  HIM* 

05  06  07  0 

8  09 

00%  minimum  081317 


A  »  Distributions  Case*Manylslands-noBlueShips 
A  ’  %MerchantClassified 


-  h — m— 


41 


0  7  0  75  0  8  0.81 


k 


100  0%  manmum  0  97967  Uean 


99  5% 
97  5% 
90  0% 
750% 
50  0% 
250% 
100% 
25% 
0  5% 
00% 


096791 
094523 
092407 
quartle  090392 
meaan  0  88043 
quartil#  085998 
083765 
080372 
0  72532 
minimum  0  71952 


-  Distributions  Cas*«Manylslands-noUAV 
a  *  V.MerchantClassified 

A  Quantiles 


99  5% 
97  5% 
900% 
750% 
500% 
25  0% 
100% 
25% 
05% 
00% 


Summary  Statistics 

0  8796045 
SMOev  00363917 

SM  Err  Uean  00016275 

Upper  95%  Uean  08828021 
Lower  95%  Mean  0  876407 


N 


500 


100  0%  manmum  093063 


092734 
0  90605 
0  88124 
quartil#  084304 
meaan  0  78478 
quamie  0  72323 
0  67945 
061737 
0  5438 
minimum  0  48855 


•  -  Summary  Statistics 
Uean  0  7804901 

SM  Dev  0  078459 

SM  Err  Uean  00035088 

Upper  95%  Uean  0  7873839 
Lower  95%  Uean  07735963 


N 


500 


Figure  37.  Distributions  for  MOE  2  for  the  500  initial  runs  across  the  four  cases  in 

the  “Many  Islands”  model. 
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The  average  percentage  of  time  the  targets  were  identified  is 
shown  in  Figure  38.  Concerning  the  Red  Ships,  there  is  no  big  difference 
whether  Blue  Ships  operate  or  not,  given  that  two  UAVs  are  deployed.  It  is  worth 
noticing  that  the  results  are  consistent  for  the  case  of  Merchant  Vessels,  too, 
because  the  surveillance  system  performs  similarly  as  before.  Additionally,  in 
every  case  the  Red  Ships  succeed  in  being  detected  fewer  times  than  the 
Merchant  Vessels  using  specific  tactics  to  avoid  the  adversary  sensors.  This  set 
of  results  provides  an  element  of  force  validation  to  the  simulation  model. 


Figure  38.  Average  percentage  of  time  the  targets  are  detected  across  the  four 
cases  of  the  “Many  Islands”  model.  (Standard  Error  ranges  from 
0.0012  to  0.0035  and  would  be  too  small  to  be  easily  seen  on  the 

bars). 


b.  Hypotheses  Testing 

One  of  the  primary  questions  of  this  thesis  is  whether  UAVs  are 
able  to  mitigate  the  incapability  of  the  current  surveillance  system  to  detect 
targets  in  complex  maritime  environments,  given  that  the  target's  tactic  is  to  take 
advantage  of  the  nearby  shorelines.  In  other  words,  whether  adding  UAVs  in  a 
surveillance  system  that  uses  traditional  units  such  as  ships  and  radars,  it 
becomes  more  sufficient.  The  hypothesis  tested  is: 
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•  H0:  there  is  no  difference  in  the  surveillance  system 
performance  whether  it  uses  UAVs  or  not,  given  that  the 
target’s  tactic  is  to  take  advantage  of  the  nearby  shorelines 
(pi=p2=M3=M4  where  the  four  populations  refer  to  the  four 
different  cases). 

•  Ha:  a  surveillance  system  with  UAVs  performs  better  than 
without  UAVs,  given  that  the  target’s  tactic  is  to  take 
advantage  of  the  nearby  shorelines  (pi^p2^M35£M4  where  the 
four  populations  refer  to  the  four  different  cases). 

To  test  the  hypothesis  above,  the  Analysis  of  Variance  (ANOVA) 
was  used.  This  parametric  method  demands  particular  assumptions,  such  as 
independence  of  groups,  the  equal  variance  assumption  and  the  Normal 
Population  assumption.  The  study  provided  in  Appendix  C  proves  that  the 
assumptions  are  met. 

Results  indicated  the  null  hypothesis  was  rejected.  The  ANOVA 
( R2=  0.861)  showed  that  there  is  a  significant  difference  between  the  four  cases 
(F(3,  1 996)=41 37.9,  p-value<  0.001)  (Figure  39).  That  is,  the  number  of  UAVs 
has  significant  effect  to  the  performance  of  the  surveillance  system. 


^  Oneway  Anova 
^  Summary  of  Fit 


Rsquare 

0.861483 

Adj  Rsquare 

0.861275 

Root  Mean  Square  Error 

0.047343 

Mean  of  Response 

0.680873 

Observations  (or  Sum  Wgts) 

2000 

Analysis  of  Variance 

Sum  of 

Source 

DF 

Squares 

Mean  Square 

F  Ratio  Prob  >  F 

Case 

3 

27.823305 

9.27444 

4137.935  <0001* 

Error 

1996 

4.473674 

0.00224 

C.  Total 

1999 

32.296979 

Figure  39.  ANOVA  for  the  initial  runs  of  the  “Many  Islands”  model. 
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To  determine  which  UAV  cases  were  significantly  different  from 
each  other  in  terms  of  the  mean  proportion  of  time  that  Red  Ships  are  positively 
identified,  multiple  comparisons  methods  were  employed  (two  sample  t  tests).  All 
the  pairwise  differences  after  conducting  Student’s  f-test  are  shown  in  Table  15. 
We  notice  that,  statistically,  every  case  differs  from  each  other.  Furthermore,  the 
largest  “improvement”  occurs  when  we  go  from  the  current  system  (no-UAV)  to  a 
solution  with  one  or  even  two  UAVs. 

Table  15.  Ordered  pairwise  differences  report  after  Student’s  f-test  (“Many 

Islands”). 


/lOrdered  Differences  Report 

Level 

•Level 

Difference  StdErrDif  Lower  CL  UpperCL  p-Value 

Many1slands-2UAVs 

Man)1slands-noUAV 

0.2952550  0.0029942  0.2893829  0,3011271  <0001' 

F,lany1slands-noBlueShips  Man^lslands-nollAV 

0.2743301  0.0029942  0.2684580  0.2802022  <0001* 

ManjIM-ILI 

ManylsIands-noUAV 

0.2323898  0.0029942  0.2265177  0.2382619  <0001’ 

Many1slands-2UAVs 

Manylslands-ILJAV 

0.0628652  0.0029942  0.0569931  0.0687373  <0001' 

Man.lsIands-noBloeShips 

Manylslands-IUAV 

0.0419403  0.0029942  0.0360682  0.0478124  <0001' 

Many1slands-2UAVs 

ManylsIands-noBlueShips  0.0209248  0.0029942  0.0150527  0.0267959  <.0001' 

1  /•••••••  • 

C.  EXPLORATORY  ANALYSIS 

The  purpose  of  this  analysis  is  not  to  predict  the  time  that  on  average  the 
targets  will  be  identified,  but  rather  to  understand  the  significant  factors  that  affect 
the  surveillance  system’s  ability  to  detect  and  identify  enemies.  For  this 
exploratory  analysis,  we  focused  on  MOE  1  with  one  UAV,  utilizing  Least 
Squares  Regression  and  Partition  Trees  analytic  methods.  These  two  methods 
were  then  compared.  This  comparison  was  helpful  to  recognize  the  significant 
factors  that  contribute  most  to  the  response  variable,  although  we  expect  some 
similarities  between  the  two  methods  in  terms  of  their  overall  importance. 
Interpretation  of  results  from  this  comparison  will  be  covered  in  Chapter  VI. 
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1. 


“Open  Sea”  Model 


The  “Open  Sea”  model  with  one  UAV  (and  specifically  the  northern  one), 
which  was  created  in  MANA,  used  the  appropriate  DOE  that  was  presented  in 
Chapter  V.  The  129  design  points  (DP)  replicated  250  times  each  resulting  in 
32,250  observations.  Instead  of  using  the  raw  data,  we  used  the  mean  values  of 
the  replicated  DP  to  reduce  the  calculations  and  produce  more  visually  efficient 
displays. 


a.  Regression  Analysis 

We  fitted  both  linear  and  non-linear  Least  Square  Regression 
Models.  The  advantage  of  the  former  is  that  they  can  often  approximate  complex 
processes  adequately.  The  later  can  be  implemented  in  broader  range  of 
applications  and  comes  to  fill  the  gap  when  linear  regression  is  incapable  of 
describing  complex  systems.  Additionally,  the  theory  associated  with  both 
models  is  well-developed  and  capable  of  producing  interpretable  predictions  and 
optimizations. 

The  step-wise  method  embedded  in  JMP  was  the  main  variable 
selection  strategy  because  it  creates  models  with  good  fit  to  the  data  using  as 
few  predictors  as  possible.  Furthermore,  the  alpha  level,  which  tests  the 
importance  of  each  one  of  the  predictors  included  in  the  model  in  the  presence  of 
the  rest,  was  set  to  .01  to  avoid  over-fitting.  This  procedure  is  useful  in  creating 
parsimonious  models  (Keymal,  2013). 

Finally,  the  models  are  compared  in  terms  of  R2  and  the  number  of 
factors  included  in  the  model.  R2  is  the  proportion  of  the  variability  of  the 
response  explained  by  the  regression  model.  In  the  output  that  will  be  presented 
later,  the  Adjusted  R2  is  also  shown.  This  metric  attempts  to  adjust  for  inflation  in 
R2  when  the  number  of  observations  is  small  compared  to  the  number  of  factors. 
Therefore,  in  discussing  the  results,  we  focus  on  adjusted  R2. 
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(1)  Main  Effects  Model.  First,  we  fitted  a  model  only  with 
main  effects  using  the  step-wise  method  and  alpha  level  of  0.01.  Prior  to 
examining  the  results,  we  first  checked  that  the  assumptions  of  the  model  were 
met.  Observing  the  Residual  by  Predicted  plot,  we  notice  that  the  errors  are  not 
equally  scattered  across  the  range  of  the  response  variable,  which  means  that 
the  variance  is  not  constant  (heteroscedasticity).  Although  the  Residual  by  Raw 
plot  indicates  that  the  errors  are  independent  of  each  other,  we  concluded  that 
the  above  model  did  not  meet  the  criteria  for  hypotheses  testing.  Both  the  plots 
are  shown  in  Figure  40. 

Residual  by  Row  Plot 
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55  0.00 

a. 

-0.10 


t  Effect  Details 
a  w  Durbin-Watson 
Durbin-  Number 

Watson  ofObs.  Autocorrelation  Prob<DW 

23042924  129  -0.1524  0.9557 

Figure  40.  Model  diagnostics  for  the  main  effects  model  prior  the  Box-Cox 

transformation  (“Open  Sea”). 

For  that  reason  we  transformed  the  input  data  according  to 
Box-Cox  result.  Figure  41  suggests  that  we  should  take  the  square  root  of  the 
data  instead  of  the  original  ones. 
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a  r  Box-Cox  Transformations 
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Figure  41 .  Box-Cox  graph  that  suggests  transformation  with  A=0.5  (“Open  Sea”). 


Indeed,  after  the  transformation  all  the  model  assumptions 
were  met.  The  Residual  by  Predicted  plot  shows  constant  variance  and  the 
Residual  by  Raw  plot  along  with  the  large  Durbin-Watson  p-value  (0.771)  prove 
the  independence  of  the  observations.  Finally,  the  Normal  Quantile  plot  shows 
the  Normality  of  the  errors  (Figure  42). 


A  Residual  by  Predicted  Plot 


Mean(PercentRe<TT  racked).M0  5) 
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Figure  42.  Model  diagnostics  for  the  main  effects  model  after  the  Box-Cox 

transformation  (“Open  Sea”). 


95 


The  model  we  ended  up  with  involves  10  factors  and  gives 
Adjusted  R2  value  of  0.781  (Table  16).  The  small  p-value  (<0.001 )  indicates  that 
at  least  one  of  the  predictors  included  in  the  model  has  great  impact  on  the 
response. 

Table  16.  Statistical  results  for  the  main  effects  model  after  the  Box-Cox 

transformation  (“Open  Sea”). 


a  -  Response  Mean(PercentRedTracked),A(0.5) 


i  Summary  of  Fit 

RSquare 

0  798401 

RSquare  A4j 

0  781317 

Root  Mean  Square  Error 

0051656 

Mean  of  Response 

0454564 

Observations  (or  Sum  Wgts) 

129 

a  Analysis  of  Variance 

Sum  of 


Source 

DF 

Squares 

Mean  Square 

F  Ratio 

Model 

10 

1.2469639 

0.124696 

46  7322 

Error 

118 

0  3148619 

0002668 

Prot)  >F 

C  Total 

128 

1  5618257 

<0001* 

The  factors  of  the  model  were  sorted  in  order  of  significance 
from  high  to  low  (Table  17).  The  detection  ranges  of  the  Blue  Sensors  seem  to 
be  the  most  significant,  followed  by  the  Communication  Accuracy  of  the 
Operational  Centre  and  the  Average  Time  between  Detections  of  the  Blue 
Radars.  The  Number  and  the  Speed  of  Merchant  Vessels  also  have  impact  to 
the  MOE,  along  with  the  Stealth  of  Red  Ships,  the  Average  Time  between 
Detections  of  the  UAV  and  the  Average  Time  between  Detections  of  the  Blue 
Ships.  To  give  the  direction  of  the  impact  of  each  predictor,  as  the  Detection 
Ranges,  the  Communication  Accuracy  and  the  Speed  of  the  Merchant  Vessels 
increase,  the  more  effective  the  surveillance  system  becomes.  In  contrast,  as  the 
Average  Time  between  Detections,  the  number  of  Merchant  Vessels  and  the  Red 
Stealth  decreases,  the  surveillance  system  becomes  less  effective. 
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Table  17. 


Sorted  parameter  estimates  for  the  main  effects  model  after  the 
Box-Cox  transformation  (“Open  Sea”). 


A  Sorted  Parameter  Estimates 


Term 

Radar_MaxDetRange 

BlueShips_MaxDetRange 

UAV_MaxDetRange 

OpsCentre_CommsAccuracy 

Radar_AvgTimeBweenDet 

NumberMerchVessels 

MerchVessels_Speed 

Red_Stealth 

UAV_AvgTi  m  e  Bwe  e  n  D  et 
BlueShips_AvgTimeBweenDet 


Estimate 

4.5688e-6 

8.9862e-6 

5.5045e-6 

0.0021806 

-0.000563 

-0.002235 

0.0048106 

-0.000875 

-0.000427 

-0.000195 


Std  Error 

3.908e-7 

7.817e-7 

6.255e-7 

0.000313 

0.000136 

0.0006 

0.001294 

0.00026 

0.000136 

6.739e-5 


t  Ratio 


11.69 

11.50 

8.80 

6.97 

-4.15 

-3.73 

3.72 

-3.36 

-3.14 

-2.90 


Prob>|t| 
<0001* 
<0001* 
<0001* 
<  0001* 
<0001* 
0.0003* 
0.0003* 
0.0010* 
0.0021* 
0.0045* 


(2)  Main  Effects  Model  with  Interactions.  To  achieve  a 
better  model,  we  added  all  the  two-way  interactions  to  the  previous  model.  Again 
we  conducted  model  diagnostics,  and  the  results  are  shown  in  Figure  43.  It  can 
be  easily  seen  that  all  the  model  assumptions  are  met.  The  Residual  by 
Predicted  plot  shows  constant  variance,  and  the  Residual  by  Raw  plot  along  with 
the  large  Durbin-Watson  p-value  (0.166)  proves  the  independence  of  the 
observations.  Finally,  the  Normal  Quantile  plot  shows  the  Normality  of  the  errors. 
No  further  transformation  is  needed. 
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A  Residual  by  Predicted  Plot 
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Figure  43.  Model  diagnostics  for  the  main  effects  with  interactions  model  (“Open 

Sea”). 


The  new  model  contains  13  factors  and  gives  Adjusted  R2 
value  of  0.823  (Table  18).  The  small  p-value  (<0.001 )  indicates  that  at  least  one 
of  the  predictors  included  in  the  model  has  great  impact  to  the  response. 

Table  18.  Statistical  results  for  the  main  effects  model  with  interactions 

(“Open  Sea”). 


a  -  Response  Mean(PercentRedTracked),A(0.5) 


i  Summary  of  Fit 

RSquare 

0  841691 

RSquare  Adj 

0823796 

Root  Mean  Square  Error 

0046368 

Mean  of  Response 

0454564 

Observations  (or  Sum  Wgts) 

129 

A  Analysis  of  Variance 


Source 

DF 

Sum  of 
Squares 

Mean  Square 

F  Ratio 

Model 

13 

1.3145754 

0101121 

47  0331 

Error 

115 

0  2472503 

0002150 

Proto  >  F 

C.  Total 

128 

1.5618257 

<0001* 

The  factors  of  the  model  with  interactions  were  sorted  in 
order  of  significance  from  high  to  low  (Table  19).  Comparing  to  the  linear  model, 
the  eight  most  significant  factors  are  exactly  the  same,  with  the  same  direction  of 
impact.  In  addition,  this  model  suggests  that  the  interaction  between  the  Number 
and  the  Speed  of  Merchant  Vessels,  the  Blue  Ships’  Average  Time  between 
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Detections,  the  interaction  between  the  Blue  Radar  Range  and  its  Time  Between 
Detections,  as  well  as  the  interaction  between  the  Blue  Radar’s  Detection  Range 
with  its  Average  Time  between  Detections  have  a  statistically  significant,  but  to  a 
lesser  extent,  effect  to  the  response.  Finally,  the  Red  Speed  affects  also  the 
MOE. 

Table  19.  Sorted  parameter  estimates  for  the  main  effects  model  with 

interactions  (“Open  Sea”). 


A  Sorted  Parameter  Estimates 


Term 

Radar_MaxDetRange 

BlueShips_MaxDetRange 

UAV_MaxDetRange 

OpsCentre_CommsAccuracy 

Radar_AvgTimeBweenDet 

MerchVessels_Speed 

NumberMerchVessels 

Red_Stealth 

(NumberMerchVessels-17.0078)*(MerchVessels_Speed-14.0155) 

UAV_AvgTimeBweenDet 

BlueShips_AvgTimeBweenDet 

(Radar_MaxDetRange-30000.2)*(Radar_AvgTimeBweenDet-62.5039) 

Red_Speed 


Estimate  Std  Error  t  Ratio 


4.5773e-6  3.508e-7 
8.9845e-6  7.017e-7 
5.4965e-6  5.61 5e-7 
0.002183  0.000281 
-0.000571  0.000122 
0.0048516  0.001162 
-0.002221  0.000538 
-0.000874  0.000234 
0.0005419  0.000145 
-0.000421  0.000122 
-0.000196  6.05e-5 

-3.373e-8  1.069e-8 
-0.001973  0.0007 


Prot»|t| 
<  0001* 
<.0001* 
<0001* 
<0001* 
<.0001* 
<0001* 
<0001* 
0.0003* 
0.0003’ 
0.0008* 
0.0016’ 
0.0020’ 
0.0057’ 


(3)  Model  with  Quadratic  Effects.  Finally,  we  fitted  a 
polynomial  model  to  check  whether  or  not  second  order  factors  affect  the  MOE. 
Again  we  conducted  model  diagnostics,  and  the  results  are  shown  in  Figure  44.  It 
can  be  easily  seen  that  all  the  model  assumptions  are  met.  The  Residual  by 
Predicted  plot  shows  constant  variance,  and  the  Residual  by  Raw  plot  along  with 
the  large  Durbin-Watson  p-value  (0.873)  proves  the  independence  of  the 
observations.  Finally,  the  Normal  Quantile  plot  shows  the  Normality  of  the  errors. 
No  further  transformation  is  needed. 
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A  Residual  by  Predicted  Plot 
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Figure  44.  Model  diagnostics  for  the  second  order  model  (“Open  Sea”). 


The  Second  Order  model  contains  13  factors  and  gives  an 
Adjusted  R2  value  of  0.822  (Table  20).  Compared  to  the  previous  model,  we 
notice  that  both  of  them  give  similar  values  of  Adjusted  R2  with  the  same  number 
of  terms.  The  small  p-value  (<0.001 )  indicates  that  at  least  one  of  the  predictors 
included  in  the  model  has  great  impact  on  the  response. 

Table  20.  Statistical  results  for  the  second  order  model  (“Open  Sea”). 


A  -  Response  Mean(PercentRedTracked),A(0.5) 


A  Summary  of  Fit 

RSquare  0840291 

RSquare  Adj  0  822237 

Root  Mean  Square  Error  0.046573 
Mean  of  Response  0  454564 

Observations  (or  Sum  Wgts)  129 


a  Analysis  of  Variance 


Source 

DF 

Sum  of 
Squares 

Mean  Square 

F  Ratio 

Model 

13 

1.3123883 

0.100953 

465431 

Error 

115 

02494374 

0  002169 

Prob>  F 

C.  Total 

128 

15618257 

<0001* 

The  factors  of  the  Second  Order  model  were  sorted  in  order 

of  significance  from  high  to  low  (Table  21).  Compared  to  the  previous  model,  we 

notice  that  the  seven  most  significant  factors  are  exactly  the  same.  In  addition, 

instead  of  the  two  interaction  terms  there  is  one  term  with  quadratic  effect  (Blue 
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Radar’s  Detection  Range).  Finally,  the  Speed  of  the  UAV  seems  to  affect  the 
response  variable  significantly.  Thus,  all  three  regression  models  are  consistent 
in  terms  of  which  are  the  most  influential  predictors. 

Table  21 .  Sorted  parameter  estimates  for  the  second  order  model  (“Open 

Sea”). 


A  Sorted  Parameter  Estimates 


Term 

Radar_MaxDetRange 

BlueShips_MaxDetRange 

UAV_MaxDetRange 

OpsCentre_CommsAccuracy 

Radar_AvgTimeBweenDet 

MerchVessels_Speed 

NumberMerchVessels 

(Radar_MaxDetRange-30000.2)*(Radar_MaxDetRange-30000.2) 

Red_Stealth 

UAV_AvgTimeBweenDet 

BlueShips_AvgTimeBweenDet 

Red_Speed 

UAV_Speed 


Estimate 

4.5709e-6 

8.9803e-6 

5.4885e-6 

0.0021807 

-0.000569 

0.0048326 

-0.002233 

-1.31e-10 

-0.000872 

-0.000423 

-0.000196 

-0.001975 

0.0004722 


Std  Error 

3.524e-7 
7.048 e-7 
5.64e-7 
0.000282 
0.000123 
0.001167 
0.000541 
3.39e-11 
0.000235 
0.000123 
6.076e-5 
0.000703 
0.000176 


t  Ratio 


Proto  |t| 

<.0001* 

<.0001* 

<0001* 

<.0001* 

<.0001* 

<.0001* 

<.0001* 

0.0002* 

0.0003* 

0.0008’ 

0.0016* 

0.0058’ 

0.0085* 


b.  Partition  Tree 

Next,  partition  trees  were  created  to  more  easily  understand  the 
regression  results  above.  Partition  Trees  can  be  easily  built  using  JMP.  They  are 
flexible  because  the  user  can  go  back  and  forth  by  “splitting”  or  “pruning”  them. 
They  are  also  able  to  handle  both  numeric  and  categorical  data.  Their  main 
characteristic  is  their  ability  at  fitting  jumps  or  plateaus  in  the  data,  which 
provides  the  analyst  with  an  easy-to-understand  output. 

To  construct  the  Partition  Tree  we  used  the  mean  values  of  the 

MOE1  as  the  response  variable  and  all  the  factors  as  input.  The  first  seven  splits 

occur  on  factors  that  were  found  to  be  the  most  important  during  the  Regression 

Analysis,  such  as  the  Detection  Ranges  of  all  the  Blue  Sensors,  as  well  as  the 

Radar’s  Average  Time  between  Detections.  The  exception  to  this  is  the  speed  of 

the  Red  Ships,  which  emerges  as  significant.  The  Partition  Tree  after  the  first 

seven  splits  (R2=  0.667)  is  shown  in  Figure  45.  We  observe  that  the  surveillance 

system  performs  better  (Mean=0.43,  SD=0.054)  when  the  maximum  detection 

ranges  of  the  Blue  Radar,  ships  and  UAV  are  more  than  22,188,  15,156  and 
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19,258  meters,  respectively,  and  the  speed  of  the  Red  Ships  is  less  than  21  kn. 
In  contrast,  the  surveillance  system  performs  worse  (Mean=0.077,  SD=0.03) 
when  the  corresponding  ranges  are  less  than  22,188,  16,406  and  15,938  meters. 


Figure  45.  Partition  tree  for  the  “Open  Sea”  model  after  seven  splits 


Following  that  we  created  a  Partition  Tree  with  21  splits,  which 
gave  R2  0.856.  The  values  of  R2  with  respect  to  the  number  of  splits  are  shown  in 
Figure  46. 


Split  History 


Figure  46.  Split  history  for  the  Partition  tree  of  the  “Open  Sea”  model. 
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After  sorting  the  variables  that  contribute  to  the  response  variable, 
we  get  the  result  of  Table  22.  Eight  out  of  the  ten  significant  factors  that  were 
found  after  fitting  the  model  with  the  main  effects  are  also  indicated  by  the 
Partition  Tree.  It  is  worth  noticing  that  just  the  Detection  Ranges  of  the  Blue 
Sensors  explain  about  78  percent  of  the  MOE  variability. 

Table  22.  Variable  contribution  for  the  Partition  tree  of  the  “Open  Sea”  model. 


a  Column  Contributions 


Term 

Number 
of  Splits 

SS 

Radar_MaxDetRange 

2 

0.32560017 

BlueShips_MaxDetRange 

2 

0.30547547 

UAV_M  axD  etRan  g  e 

4 

0.22704193 

Radar_AvgTimeBweenDet 

3 

0.08647983 

Red_Speed 

1 

0.05046437 

BlueShips_Speed 

1 

0.03202279 

Red_Stealth 

1 

0.02894743 

J 

UAV_AvgTi  meBweenDet 

1 

0.02296974 

J  : 

UAV_ProbClass 

2 

0.01281549 

r 

BlueShips_PrClass 

1 

0.00899291 

NumberMerchVessels 

1 

0.00993990 

MerchVessels_Speed 

1 

0.00978825 

OpCentre_CommsLatency 

1 

0.00230404 

2.  “Many  Islands”  Model 

The  “Many  Islands”  model  with  one  UAV  (and  specifically  the  northern 
one),  which  was  created  in  MANA,  used  the  appropriate  DOE  that  was  presented 
in  Chapter  V.  The  129  DP  were  replicated  500  times,  each  resulting  in  65,000 
observations.  Instead  of  using  the  raw  data,  we  use  the  mean  values  of  the 
replicated  DP  to  reduce  the  calculations  and  produce  more  visually  efficient 
displays. 
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a.  Regression  Analysis 

(1)  Main  Effects  Model.  First  we  fitted  a  model  with  only 
the  main  effects  of  all  the  factors.  Using  the  stepwise  strategy  and  setting  an 
alpha  level  of  0.01,  we  came  to  a  model  with  only  eight  terms.  We  conducted 
model  diagnostics  and  we  noticed  that  all  the  model  assumptions  are  met.  The 
Residual  by  Predicted  plot  shows  constant  variance,  and  the  Residual  by  Raw 
plot  along  with  the  large  Durbin-Watson  p-value  (0.589)  proves  the 
independence  of  the  observations.  Finally,  the  Normal  Quantile  plot  shows  the 
Normality  of  the  errors  (Figure  47). 


4  Residual  by  Predicted  Plot 


Predicted 


4  Residual  by  Row  Plot 


Row  Number 


A  ■*  Durbin-Watson 
Durbin-  Number 

Watson  ol  OOs.  Autocorrelation  Prob<DW 

20306976  129  -00167  0  5898 


A  *  Residual  Mean(PercentRedTracked)  -Linear 


Figure  47.  Model  diagnostics  for  the  main  effects  model  (“Many  Islands”). 


The  model  we  ended  up  with  involves  eight  factors  and  gives 
an  Adjusted  R2  value  of  0.742  (Table  23).  The  small  p-value  (<0.001 )  indicates 
that  at  least  one  of  the  predictors  included  in  the  model  has  great  impact  on  the 
response. 
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Table  23. 


Statistical  results  for  the  main  effects  model  (“Many  Islands”). 


3  •»  Response  Mean(PercentRedTracked) 


•d  Summary  of  Fit 

RSquare  0  75830S 

RSquareAdj  0.742192 

Root  Mean  Square  Error  0  049536 

Mean  of  Response  0  359977 

Observations  (or  Sum  Wgts)  129 


^  Analysis  of  Variance 

Sum  ol 


Source 

DF 

Squares 

Mean  Square 

F  Ratio 

Model 

8 

09238422 

0.115480 

47  0616 

Error 

120 

02944571 

0002454 

Prob  >  F 

C.  Total 

128 

12182993 

<0001* 

The  factors  of  the  model  were  sorted  in  order  of  significance 
from  high  to  low  (Table  24).  The  Communication  Accuracy  of  the  Operational 
Centre  emerges  as  the  most  significant  factor  followed  by  the  detection  ranges  of 
the  Blue  Sensors.  The  Stealth  of  the  Red  Ships  comes  next,  followed  by  the 
Average  Times  between  Detections  of  the  three  Blue  Sensors.  To  give  the 
direction  of  the  impact  of  each  predictor,  as  the  Communication  Accuracy  and 
the  Detection  Ranges  of  the  Blue  Sensors  increase,  the  more  effective  the 
surveillance  system  becomes.  In  contrast,  as  the  Red  Stealth  and  the  Average 
Time  between  Detections  decreases,  the  surveillance  system  becomes  less 
effective. 


Table  24.  Sorted  parameter  estimates  for  the  main  effects  model  (“Many 

Islands”). 


43  Sorted  Parameter  Estimates 


Term 

OpsCentre_CommsAccuracy 

BlueShips_MaxDetRange 

UAV_MaxDetectionRange 

Radar_MaxDetRange 

Red_Stealth 

UAV_AvgTi  m  e  Bwe  e  n  D  et 

Radar_AvgTimeBweenDet 

BlueShips_AvgTimeBweenDet 


Estimate 

0.0033661 
6.9471  e-6 
4.8265e-6 
2.271 5e-6 
-0.001253 
-0.000549 
-0.000438 
-0.000175 


Std  Error 

0.0003 

7.496e-7 

5.997e-7 

3.748e-7 

0.00025 

0.00013 

0.00013 

6.462e-5 


t  Ratio 

11.23 

9.27 

8.05 

6.06 

-5.02 

-4.22 

-3.36 

-2.71 


Prob>|t| 

<0001* 

<0001* 

<0001* 

<0001* 

<0001* 

<0001* 

0.0010* 

0.0076* 
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(2)  Main  Effects  Model  with  Interactions.  To  achieve  a 
better  model,  we  added  all  the  two-way  interactions  to  the  previous  model.  Again 
we  conducted  model  diagnostics  and  the  results  are  shown  in  Figure  48.  It  can 
be  easily  seen  that  all  the  model  assumptions  are  met.  The  Residual  by 
Predicted  plot  shows  constant  variance,  and  the  Residual  by  Raw  plot  along  with 
the  large  Durbin-Watson  p-value  (0.607)  proves  the  independence  of  the 
observations.  Finally,  the  Normal  Quantile  plot  shows  the  Normality  of  the  errors. 
No  further  transformation  is  needed. 


4  Residual  by  Predicted  Plot  4  t  Durbin-Watson 


Predicted 


4  Residual  by  Row  Plot 
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Figure  48.  Model  diagnostics  for  the  main  effects  model  with  interactions  (“Many 

Islands”). 


The  new  model  contains  ten  factors  and  gives  Adjusted  R2 
value  of  0.798  (Table  25).  The  small  p-value  (<0.001 )  indicates  that  at  least  one 
of  the  predictors  included  in  the  model  has  great  impact  on  the  response. 
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Table  25. 


Statistical  results  for  the  main  effects  model  with  interactions 
(“Many  Islands”). 


A  -  Response  Mean(PercentRedTracked) 
a  Summary  of  Fit  *  Analysis  of  Variance 


RSquare 

RSquare  Adj 

0814299 

0798562 

Source 

DF 

Sum  of 
Squares 

Mean  Square 

F  Ratio 

Root  Mean  Square  Error 

0043787 

Model 

10 

09920603 

0099206 

51  7431 

Mean  of  Response 

0359977 

Error 

118 

02262390 

0001917 

Proto  >  F 

Observations  (or  Sum  Wgts) 

129 

C.  Total 

128 

1.2182993 

<0001* 

The  factors  of  the  model  with  interactions  were  sorted  in 
order  of  significance  from  high  to  low  (Table  26).  This  model  involves  all  the  eight 
factors  that  the  previous  model  had.  In  addition,  the  six  most  significant  factors 
are  exactly  the  same.  Furthermore,  this  model  suggests  that  the  interactions 
between  the  Operational  Centre’s  Communication  Accuracy  with  UAV’s 
Detection  Range  and  the  Stealth  of  the  Red  ships  are  also  significant. 

Table  26.  Sorted  parameter  estimates  for  the  main  effects  model  with 

interactions  (“Many  Islands”). 


a  Sorted  Parameter  Estimates 


Term 

OpsCentre_CommsAccuracy 

BlueShips_MaxDetRange 

UAV_MaxDetectionRange 

Radar_MaxDetRange 

Red_Stealth 

UAV_AvgTimeBweenDet 

(UAV_MaxDetedionRange-17500)*(OpsCentre_CommsAccuracy-75.0078) 
(OpsCentre_CommsAccuracy-75.0078)*(Red_Stealth-60  0155) 
Radar_AvgTimeBweenDet 
BlueShips_AvgTimeBweenDet 


Estimate  Std  Error  t  Ratio 


0.0033653  0.000265 
6.947e-6  6.626e-7 
4.8268e-6  5.301  e-7 
2.2708e-6  3.31 3e-7 
-0.001255  0.000221 
-0.000549  0.000115 
1.6023e-7  3.499e-8 
-0.000059  1.533e-5 
-0.000438  0.000115 
-0.000176  5.712e-5 


12.70 

10.48 

9.11 

6.85 

-5.69 

-4.77 

4.58 

-3.85 

-3.80 

-3.07 


Proto  |t| 
<0001* 
<0001* 
<0001* 
<0001* 
<0001* 
<0001* 
<0001* 
0.0002* 
0.0002’ 
0.0026* 


(3)  Model  with  Ouadratic  Effects.  Finally,  we  fitted  a 
polynomial  model  to  check  whether  or  not  second  order  factors  affect  the  MOE. 
Again,  we  conducted  model  diagnostics,  and  the  results  are  shown  in  Figure  49. 
It  can  be  easily  seen  that  all  the  model  assumptions  are  met.  The  Residual  by 
Predicted  plot  shows  constant  variance,  and  the  Residual  by  Raw  plot  along  with 
the  large  Durbin-Watson  p-value  (0.357)  proves  the  independence  of  the 
observations.  Finally,  the  Normal  Ouantile  plot  shows  the  Normality  of  the  errors. 
No  further  transformation  is  needed. 
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4>  Residual  by  Predicted  Plot  ^  „  Durbin-Watson 


Figure  49.  Model  diagnostics  for  second  order  model  (“Many  Islands”). 


The  last  model  contains  12  factors  and  gives  Adjusted  R2 
value  of  0.826  (Table  27).  The  small  p-value  (<0.001 )  indicates  that  at  least  one 
of  the  predictors  included  in  the  model  has  great  impact  on  the  response. 

Table  27.  Statistical  results  for  the  second  order  model  (“Many  Islands”). 


^  -  Response  Mean(PercentRedTracked) 


i  Summary  of  Fit 

RSquare 

0842367 

RSquare  Adj 

082606 

Root  Mean  Square  Error 

0040689 

Mean  of  Response 

0359977 

Observations  (or  Sum  Wgts) 

129 

a  Analysis  of  Variance 

Sum  of 


Source 

OF 

Squares 

Mean  Square 

F  Ratio 

Model 

12 

10262547 

0085521 

51  6571 

Error 

116 

01920446 

0001656 

Prob  >  F 

C.  Total 

128 

12182993 

<0001* 

The  factors  of  the  model  with  interactions  were  sorted  in 
order  of  significance  from  high  to  low  (Table  28).  Comparing  to  the  previous 
models,  we  notice  that  this  contains  all  the  main  effects  we  observed  previously, 
but  instead  of  the  interaction  terms,  it  involves  the  quadratic  effects  of  the 
Communication  Accuracy  of  the  Operational  Centre,  the  Detection  Range  of  the 
Blue  Radar  and  the  UAV,  as  well  as  the  Time  between  detections  of  the  UAV. 
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Table  28. 


Sorted  parameter  estimates  for  the  second  order  model  (“Many 

Islands”). 


A  Sorted  Parameter  Estimates 


Term 

OpsCentre_CommsAccuracy 

BlueShips_MaxDetRange 

UAV_MaxDetectionRange 

Radar_MaxDetRange 

Red_Stealth 

UAV_AvgTimeBweenDet 

Radar_AvgTimeBweenDet 

(OpsCentre_CommsAccuracy-75.0078)‘(OpsCentre_CommsAccuracy-75.0078) 

(Radar_MaxDetRange-30000.2)*(Radar_MaxDetRange-30000.2) 

BlueShips_AvgTimeBweenDet 

(UAV_AvgTimeBweenDet-62.5039)’(UAV_AvgTimeBweenDet-62.5039) 

(UAV_MaxDetedionRange-17500)’(UAV_MaxDetectionRange-17500) 


Estimate  Std  Error  t  Ratio 

0.003368  0.000246  13.68 

6.9497e-6  6.157e-7  11.29 

4.823e-6  4.926e-7  9.79 

2.273e-6  3.079e-7  7.38 

-0.001253  0.000205  -6.11 

-0.000549  0.000107  -5.13 

-0.000438  0.000107  -4.09 

8.042e-5  0.00002  4.00 

-1.12e-10  3.18e-11  -3.52 

-0.000175  0.000053  -3.30 

1.205e-5  3.661  e-6  3.29 

-2.03e-10  7.89e-11  -2.58 


Prob>|t| 

<0001* 

<0001* 

<0001* 

<0001* 

<0001* 

<0001* 

<0001* 

0.0001’ 

0.0006’ 

0.0013* 

0.0013* 

0.0112’ 


b.  Partition  Tree 

To  construct  the  Partition  Tree,  we  used  the  mean  values  of  the 
MOE1,  as  the  response  variable,  and  all  the  factors  as  input.  The  first  six  splits 
occur  on  factors  that  found  to  be  the  most  important  during  the  Regression 
Analysis,  such  as  the  Communication  Accuracy  of  the  Operational  Centre  and 
Detection  Ranges  of  all  the  Blue  Sensors.  The  Partition  Tree  after  the  first  six 
splits  ( R2=  0.52)  is  shown  in  Figure  50.  We  observe  that  the  surveillance  system 
performs  better  (Mean=0.49,  SD=0.07)  when  the  Communication  Accuracy  of  the 
Operational  Centre  is  greater  than  78  percent  and  the  Blue  Ships’  Detection 
Range  is  more  than  19,844  meters.  In  contrast,  the  surveillance  system  performs 
worse  (Mean=0.25,  SD=0.05)  when  the  corresponding  values  are  less  than78 
percent  and  1 1 ,904  meters,  respectively. 
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Figure  50.  Partition  tree  for  the  “Many  Islands”  model  after  six  splits. 


Following  that  we  created  a  Partition  Tree  with  20  splits  which  gave 
R2  0.821.  The  values  of  R2  with  respect  to  the  number  of  splits  are  shown  in 
Figure  51 . 


Figure  51 .  Split  history  for  the  Partition  tree  of  the  “Many  Islands”  model. 

After  sorting  the  variables  that  contribute  to  the  response  variable, 
we  get  the  result  of  Table  29.  It  is  worth  noticing  that  the  six  most  significant 
terms  that  the  Partition  Tree  indicates  explain  about  94  percent  of  the  MOE 
variability  and  were  also  predictors  in  the  Regression  Analysis. 
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Table  29. 


Variable  contribution  for  the  Partition  tree  of  the  “Many  Islands” 

model. 


a  Column  Contributions 


Term 

Number 
of  Splits 

SS 

OpsCentre_CommsAccuracy 

4 

0.35727677 

BlueShips_MaxDetRange 

4 

0.31591995 

UAV_M  axD  etectionRange 

4 

0.12535109 

UAV_AvgTi  meBweenDet 

1 

0.06931791 

Radar_MaxDetRange 

1 

0.04084012 

Red_Stealth 

1 

0.02735960 

OpCentre_CommsLatency 

1 

0.02431740 

BlueShips_PrClass 

1 

0.02071166 

UAV_CommsLatency 

1 

0.00881418 

UAV_ProbClass 

1 

0.00864855 

Radar  ProbClass 

1 

0.00140383 
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VI.  CONCLUSION 


A.  DISCUSSION  AND  OPERATIONAL  INSIGHTS 

1.  Answers  to  the  Primary  Research  Question 

The  hypothesis  we  tested  was  whether  “a  surveillance  system  with  UAVs 
performs  better  than  one  without  UAVs  assuming  that  the  target’s  tactic  is  to 
approach  transiting  merchant  vessels  or  to  take  advantage  of  nearby  shorelines.  ” 
This  hypothesis  was  confirmed.  The  simulation  we  created  shows  that 
integrating  one  or  two  UAVs  into  a  traditional  surveillance  system  makes  it  more 
efficient  in  the  detection  and  persistent  surveillance  of  enemies  and  neutral 
targets.  This  conclusion  applies  to  all  the  areas  of  the  Aegean  Sea,  either  the 
Northern  and  Southern  Aegean  Sea  (modeled  by  the  “Open  Sea”  model),  or  the 
Central  and  Southeast  (modeled  by  the  “Many  Islands”  model). 

In  all  the  cases  we  studied,  we  found  that  a  traditional  system  comprised 
only  of  surface  ships  and  ground  radars  performs  worse  than  a  system  that  uses 
UAVs.  Additionally,  we  explored  a  “futuristic”  scenario  where  only  two  UAVs 
support  the  ground  radars,  without  the  use  of  surface  ships  (no  Blue  Ships 
scenario).  The  results  showed  that  we  can  obtain  similar  results  with  this  system 
as  with  the  system  in  which  we  deploy  one  UAV  and  two  surface  ships  in  support 
of  the  ground  radars  (1-UAV  scenario).  This  conclusion  suggests  that  the 
decision  makers  should  review  the  surveillance  policy.  They  can  shift  to  less 
reliance  on  the  cost-ineffective  use  of  surface  ships  towards  the  increased  use  of 
UAVs  to  better  monitor  maritime  areas.  This  solution  might  be  more  feasible  in 
adverse  weather  conditions  and  when  there  is  lack  of  operational  surface  units 
available. 

This  study  also  demonstrated  that  the  potential  tactics  the  enemy  might 
use  to  avoid  detection  cannot  mislead  UAVs.  Regardless  of  whether  the  enemies 
try  to  take  advantage  of  the  nearby  merchant  vessels  and  sail  close  to  them  to 
transit  long  distances,  or  they  use  the  shorelines  of  numerous  islands  as 
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coverage,  a  surveillance  system  that  deploys  UAVs  has  significantly  better 
performance  than  a  system  that  uses  only  ground  radars  and  surface  ships. 

This  thesis  also  shows  that  the  area  from  where  a  UAV  is  launched  affects 
the  performance  of  the  whole  surveillance  system.  As  it  is  easily  understood,  the 
more  contacts  there  are  close  to  the  launch  site,  the  more  the  UAV  contributes  to 
the  surveillance  system.  That  is,  if  the  launch  site  is  close  to  areas  with  high 
traffic,  the  UAV  will  have  more  opportunities  to  locate  the  targets.  Additionally, 
the  targets  will  be  identified  earlier.  Therefore,  it  is  extremely  important  to  define 
the  best  launch  site  for  the  UAVs.  From  the  literature  review,  we  found  that  UAVs 
can  be  launched  either  from  ground-based  sites  or  appropriately  equipped 
surface  ships.  Both  solutions  must  be  examined  for  strengths  and  vulnerabilities. 

Furthermore,  this  study  showed  that  the  UAVs  can  mitigate  a  vulnerability 
that  the  ground  radar  systems  inherently  have:  the  incapability  to  cover  their 
“blind  sectors”  that  result  from  the  way  systems  must  be  positioned.  Because 
monitoring  the  whole  range  of  the  AOI  with  ground  radars  is  not  practical  (it 
would  require  twice  as  many  sensors),  the  existing  radars  are  placed  in  areas 
with  the  most  traffic.  As  a  consequence,  these  systems  are  not  always  stationed 
at  the  highest  point  of  an  island,  which  results  in  a  limited  field  of  view. 
Additionally,  taking  into  account  that  radar  technology  is  subject  to  technical 
restrictions,  the  detecting  capabilities  are  further  narrowed.  UAVs  can  easily 
cover  those  areas,  providing  information  about  existing  targets. 

Finally,  it  is  worth  noticing  that  the  MOE  in  this  study  was  the  percentage 
of  time  the  targets  are  positively  identified.  If  another  MOE  had  been  set,  i.e.,  the 
time  until  the  first  detection  or  classification,  then  the  value  of  the  use  of  UAVs  is 
expected  to  be  further  recognized.  Their  speed  and  their  potential  larger 
detection  ranges  make  them  the  most  valuable  assets  compared  to  all  the  other 
traditional  surveillance  units. 


114 


2.  Answers  to  the  Exploratory  Research  Question 

This  thesis  found  the  important  factors  that  contribute  to  the  maritime 
surveillance  system.  The  two  models,  which  simulated  corresponding 
environments  in  terms  of  transiting  merchant  vessels,  the  existence  of  nearby 
islands  and  an  enemy’s  tactics,  were  studied  separately,  but  most  of  the  input 
variables  appear  in  both  of  them.  These  factors  are  described  below. 

•  The  detection  range  of  all  the  sensors  that  are  part  of  the 
surveillance  system  seems  to  have  the  most  significant  positive 
effect  on  the  percentage  of  time  the  targets  are  classified.  The 
partition  tree  for  the  “Open  Sea”  model  highlights  the  importance  of 
having  the  radars,  the  UAV  and  the  ships  detection  ranges  larger 
than  22,  19  and  16  km,  respectively. 

•  Communication  Accuracy  of  the  Operational  Centre  has  significant 
positive  effect  on  the  system’s  performance.  The  partition  tree  for 
the  “Many  Islands”  model  suggests  if  this  factor  has  a  value  of  at 
least  78  percent,  it  will  have  an  ability  to  better  monitor  the  area  of 
interest.  It  is  worth  noticing  that  the  Operational  Centre  is  crucial  in 
the  communication  network  that  we  established  because  its  role  is 
to  gather  the  information  from  each  one  of  the  sensors  and  send 
them  to  the  patrolling  units. 

•  Enemy’s  stealth  is  a  factor  that  has  significant  negative  impact  on 
the  percentage  of  time  it  is  being  classified.  Enemy’s  stealth  is 
presented  in  both  the  scenarios  we  studied  where  a  target  used  a 
specific  tactic  to  avoid  being  detected  by  the  surveillance  system.  In 
reality,  this  factor  describes  the  enemy’s  attempt  to  avoid  the  visual, 
IR  and  radar’s  contact  using  various  methods. 

•  The  number  of  merchant  vessels  and  their  speed  has  a  negative 
effect  on  the  surveillance  system’s  performance  only  in  the  case  in 
which  the  enemy  tries  to  use  them  as  a  “Trojan  Horse”  to  transit 
long  distances.  In  environments  where  there  is  high  traffic,  either 
from  merchant  or  fishing  vessels,  but  the  enemy  does  not  try  to 
take  advantage  of  it,  it  seems  that  the  surveillance  system  is  not 
affected. 

•  Average  Times  between  Detections  of  our  own  sensors  are  some 
more  important  factors  in  terms  of  the  surveillance  system’s 
detection  capability.  All  of  them  positively  affect  the  percentage  of 
time  the  targets  are  classified  regardless  the  operational 
environment  the  system  operates  in. 

•  Enemy’s  speed  is  a  factor  that  has  a  significant  negative  effect  on 
the  percentage  of  time  the  targets  are  classified.  This  factor 
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emerged  only  in  the  “Open  Sea”  model  where  the  target  has  to 
transit  long  distances  being  “exposed”  to  adversary’s  surveillance 
system.  This  is  operationally  reasonable  because  when  a  target 
sails  very  close  to  the  shoreline  to  avoid  the  detection,  it  cannot 
maintain  a  high  speed  due  to  navigation  dangers. 

•  UAV  speed  has  a  positive  effect  on  the  system’s  performance  only 
in  the  case  where  it  has  to  cover  large  areas.  This  study  showed 
that  when  the  UAV  had  to  cover  areas  with  large  number  of  islands, 
its  speed  was  not  important. 

B.  FUTURE  RESEARCH 

This  study  was  focused  on  the  effectiveness  of  UAVs  when  integrated  in  a 
traditional  surveillance  system  that  monitors  a  complex  maritime  environment.  By 
expanding  the  field  of  research  or  focusing  on  other  key  points,  the  following 
topics  might  be  explored  by  future  studies: 

•  The  requirements  for  a  surveillance  system  change  significantly 
based  on  the  situation  in  which  it  operates.  Do  we  need  it  during 
peace  time  or  war?  How  do  things  change  if  the  UAV  is  capable  of 
strike  operations  and/or  it  is  subject  to  being  hit  by  enemy  targets? 
How  vulnerable  are  the  ground  radars  to  attacks?  If  they  are  hit, 
how  much  of  the  whole  system’s  performance  is  affected?  A  more 
war-focused  scenario  is  believed  to  be  able  to  provide  insights  to 
the  aforementioned  questions. 

•  This  study  explored  two  potential  tactics  that  enemies  could  have 
used  in  a  particular  environment.  They  aimed  to  approach  either 
large  transiting  merchant  vessels  or  shorelines  of  the  nearby 
islands.  A  combination  of  those,  along  with  other  enemy  tactics, 
might  be  further  explored. 

•  Given  the  complexity  of  the  Aegean  Sea  and  the  number/type  of 
assets  assigned,  a  study  could  indicate  the  optimal  placement  of 
the  ground  radars.  Additionally,  should  UAVs  be  launched  from 
particular  ground-based  sites  or  from  command  ships  that  can  be 
deployed  wherever  the  demands  are? 

•  A  more  detailed  study  could  be  done  on  the  cost  implications  of  the 
sensors  that  participate  in  the  surveillance  system.  Is  it  cost- 
effective  to  use  UAVs  or  surface  ships?  How  many 
UAVs/ships/ground  radars  should  be  used?  The  financial  problems 
and  the  budget  constraints  that  most  navies  have  to  face  make 
these  questions  more  challenging. 
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C.  TAKEAWAY 

This  thesis  demonstrates  that  by  integrating  one  or  two  UAVs  into  a 
traditional  surveillance  system,  the  system  becomes  more  efficient  in  constantly 
detecting  enemies  or  neutral  targets  regardless  of  the  complexity  of  the  maritime 
environment  in  which  it  operates.  Additionally,  the  potential  tactics  the  enemy 
surface  ships  might  use  to  avoid  detection  cannot  mislead  UAVs.  This  study 
showed  also  that  the  area  from  where  a  UAV  is  launched  affects  the 
performance  of  the  whole  surveillance  system;  in  particular,  UAVs  can  mitigate 
the  surveillance  system’s  inability  to  cover  its  “blind  sectors,”  which  are  inherent 
in  ground  radar  systems.  Finally,  the  most  important  factors  that  affect  the 
surveillance  system’s  performance  in  all  the  areas  of  Aegean  Sea  are  the 
detection  range  of  all  the  sensors,  the  communication  accuracy  of  the 
Operational  Centre,  an  enemy’s  stealth  and  the  average  times  between 
detections  of  our  own  sensors.  Moreover,  in  areas  where  the  enemy  has  to 
transit  long  distances  without  the  existence  of  many  islands  nearby,  the  number 
of  merchant  vessels  and  their  speed,  along  with  the  enemy’s  speed  and  UAV 
speed,  emerge  as  factors  that  have  a  significant  effect  on  the  percentage  of  time 
the  targets  are  classified. 
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APPENDIX  A.  COMPARISON  BETWEEN  NORTHERN  AND 
SOUTHERN  UAV  FOR  BOTH  OPEN  SEA  AND  MANY  ISLANDS 

MODELS 


In  the  following  sections  we  provide  a  detailed  comparison  between  the 
two  different  options  when  only  one  UAV  is  deployed.  In  the  first  scenario,  the 
UAV  is  launched  from  a  location  in  the  Northwest,  whereas  in  the  second,  the 
UAV  started  its  deployment  from  the  Southeast  (“Open  Sea”  model)  or  the 
Southwest  (“Many  Islands”  model),  respectively.  The  information  that  follows 
involves  boxplots,  ANOVA  and  t- Test  results.  First,  information  for  the  “Open 
Sea”  model  is  presented.  In  both  the  cases,  statistical  analysis  shows  that  the 
location  from  which  the  UAV  is  launched  does  matter, 
a.  “Open  Sea”  model. 


t  Test 

OpenSea-1  UAV(South)-OpenSea-1  UAV  (North) 
Assuming  equal  variances 


Difference 
Std  Err  Dif 
Upper  CL  Dif 
Lower  CL  Dif 
Confidence 


-0.03376  t  Ratio 
0.00360  DF 
-0.02670  Prob  >  |t| 
-0.04082  Prob  >  t 
0.95  Prob  <  t 


-9.38131 

998 

<.0001* 

1.0000 

<.0001* 
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Analysis  of  Variance 


Source 

DF 

Sum  of  Squares 

Mean  Square 

F  Ratio 

Prob  >  F 

Case 

1 

0.2849663 

0.284966 

88.0090 

<.0001* 

Error 

998 

3.2314459 

0.003238 

C.  Total 

999 

3.5164122 

Means  for  Oneway  Anova 

Level  Number 


OpenSea-lUAV  (North) 
OpenSea-1  UAV(South) 


500 

500 


Mean  Std  Error 

0.603269  0.00254 

0.569507  0.00254 


Std  Error  uses  a  pooled  estimate  of  error  variance 


Lower  95% 

0.59828 

0.56451 


Upper  95% 

0.60826 

0.57450 


b.  “Many  Islands”  model 
Oneway  Analysis  of  %RedClassified  By  Case 


Case 


t  Test 

Manylslands-1  UAV(South)-Manylslands-1  UAV  (North) 

Assuming  equal  variances 

5.24457 
998 

<.0001* 
1.0000 
<.0001* 


Difference 
Std  Err  Dif 
Upper  CL  Dif 
Lower  CL  Dif 
Confidence 


-0.01981  t  Ratio 
0.00378  DF 
-0.01240  Prob  >  |t| 
-0.02723  Prob  >  t 
0.95  Prob  <  t 
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Analysis  of  Variance 


Source 

DF 

Sum  of  Squares 

Mean  Square 

F  Ratio 

Prob  >  F 

Case 

1 

0.0981541 

0.098154 

27.5055 

<.0001* 

Error 

998 

3.5613916 

0.003569 

C.  Total 

999 

3.6595457 

Means  for  Oneway  Anova 

Level  Number 

Manylslands-1  UAV  (North)  500 

Manylslands-1  UAV(South)  500 


Mean 

0.722677 

0.702862 


Std  Error 

0.00267 

0.00267 


Lower  95% 

0.71743 

0.69762 


Upper  95% 

0.72792 

0.70810 


Std  Error  uses  a  pooled  estimate  of  error  variance. 
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APPENDIX  B.  DIAGNOSTICS  FOR  PARAMETRIC  ASSUMPTIONS 
OF  MOE  2  IN  OPEN  SEA  MODEL 


Our  decision  whether  to  use  a  parametric  or  non-parametric  method  for 
the  analysis  is  based  on  whether  the  assumptions  of  independence,  equality  of 
variance,  and  normality  are  met.  Below  we  study  those  assumptions  for  the 
“Open  Sea”  model  with  respect  to  MOE  2.  The  boxplots  and  the  Histogram  of 
Residuals  below  show  that  the  Equal  Variance  Assumption  and  the  Near 
Normality  Assumption,  respectively,  hold. 
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APPENDIX  C.  DIAGNOSTICS  FOR  PARAMETRIC  ASSUMPTIONS 
OF  BOTH  MOES  IN  MANY  ISLANDS  MODEL 


In  this  appendix  we  provide  the  results  of  our  study  on  whether  the 
assumptions  of  independence,  equality  of  variance,  and  normality  are  met.  The 
results  refer  to  both  MOEs  of  the  “Many  Islands”  model.  First,  the  results  about 
MOE1  are  presented.  The  boxplots  and  the  Histogram  of  Residuals  that  follow 
show  that  the  Equal  Variance  Assumption  and  the  Near  Normality  Assumption, 
respectively,  hold. 

a.  MOE  1 
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b.  MOE  2 
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